ProvSQL C/C++ API
Adding support for provenance and uncertainty management to PostgreSQL databases
Loading...
Searching...
No Matches
provsql_rmgr.c File Reference

WAL-logging the circuit store through a custom resource manager. More...

#include "postgres.h"
#include "provsql_config.h"
#include "access/xlog.h"
#include "access/xlog_internal.h"
#include "access/xloginsert.h"
#include "access/xlogreader.h"
#include "access/xlogrecord.h"
#include "miscadmin.h"
#include "utils/builtins.h"
#include "provsql_mmap.h"
#include "provsql_rmgr.h"
#include "provsql_shmem.h"
#include "provsql_utils.h"
Include dependency graph for provsql_rmgr.c:

Go to the source code of this file.

Macros

#define RM_PROVSQL_ID   151
 Resource-manager id.
#define XLOG_PROVSQL_STORE   0x10
 The only record kind: a store message to replay verbatim.

Functions

static void provsql_rmgr_redo (XLogReaderState *record)
static void provsql_rmgr_cleanup (void)
 End of WAL replay: the replay spool reaches the disk before the end-of-recovery checkpoint.
static void provsql_rmgr_desc (StringInfo buf, XLogReaderState *record)
static const char * provsql_rmgr_identify (uint8 info)
void provsql_register_rmgr (void)
 Register ProvSQL's resource manager with PostgreSQL.
bool provsql_store_write_allowed (void)
 Whether this process may write to the circuit store.
void provsql_wal_log_store_message (const char *data, size_t len)
 Write one store message to the WAL, if WAL logging is on.

Variables

bool provsql_store_wal_logging = false
 Global variable set by the provsql.store_wal_logging run-time configuration parameter: when true, every mutation of the circuit store is written to the WAL before it is written to the store.
static bool in_redo = false
 True while this process is replaying a ProvSQL WAL record, which is the one case where writing to the store during recovery is right.
static const RmgrData provsql_rmgr

Detailed Description

WAL-logging the circuit store through a custom resource manager.

The circuit store is not a set of relations, so PostgreSQL's own machinery does not carry it: it is invisible to crash recovery, streaming replication and PITR. From PostgreSQL 15 an extension can register a resource manager of its own and write WAL records the startup process will replay, which is the one mechanism that brings a non-relation file under WAL without putting it in shared buffers.

The design is the one the store's shape makes natural. Every mutation is already a self-describing message – opcode, database, payload – so a record is just that message, and replay is feeding it back to the worker. Records are not tied to the commit: like an index page split they are applied whether or not the transaction commits, which is exactly the semantics gate creation already has, and replay is idempotent because creating a gate that exists is a no-op and writing a probability a gate already holds is a no-op too.

What it is for. Streaming replication and PITR: a standby's startup process applies these records to its own store, so a replica can carry provenance. Crash recovery does not depend on them, and that is deliberate: provsql.store_wal_logging requires provsql.store_synchronous_commit, so a transaction cannot commit until its store writes are on disk, and the store on disk is therefore never behind the WAL. That requirement is what removes the checkpoint gap an asynchronous store would open – WAL before the last checkpoint's redo pointer is recycled, so a store that lagged across a whole checkpoint could not be repaired by replay.

What it is not. A hot-standby backend must not write to the store, so provenance queries on a standby only work for gates that already exist there – and almost every provenance query creates gates, reads included. Making them work needs a session-local overlay store, which is a different piece of work.

Off by default: it changes what a cluster writes to its WAL, and a replica that has never seen these records is better off without them.

Definition in file provsql_rmgr.c.

Macro Definition Documentation

◆ RM_PROVSQL_ID

#define RM_PROVSQL_ID   151

Resource-manager id.

PostgreSQL reserves 128-255 for extensions, and one cluster cannot load two extensions claiming the same id: replay would hand one extension's records to the other. Which ids are taken is kept at https://wiki.postgresql.org/wiki/CustomWALResourceManagers, where 151 is reserved for ProvSQL; keep this in step with that page.

RM_EXPERIMENTAL_ID (128) is what the page asks unreleased work to use, and is where this started; a released extension takes an id of its own.

Definition at line 79 of file provsql_rmgr.c.

◆ XLOG_PROVSQL_STORE

#define XLOG_PROVSQL_STORE   0x10

The only record kind: a store message to replay verbatim.

The low four bits of the info byte belong to PostgreSQL, so the tag lives in the high nibble; the message's own opcode is the first byte of the payload, which is all replay needs.

Definition at line 86 of file provsql_rmgr.c.

Function Documentation

◆ provsql_register_rmgr()

void provsql_register_rmgr ( void )

Register ProvSQL's resource manager with PostgreSQL.

Must be called from _PG_init under shared_preload_libraries, and is called unconditionally – registration is what lets any WAL that already contains ProvSQL records be replayed, whether or not this cluster is currently writing them. A no-op before PostgreSQL 15.

Definition at line 146 of file provsql_rmgr.c.

Here is the caller graph for this function:

◆ provsql_rmgr_cleanup()

void provsql_rmgr_cleanup ( void )
static

End of WAL replay: the replay spool reaches the disk before the end-of-recovery checkpoint.

Definition at line 112 of file provsql_rmgr.c.

Here is the call graph for this function:

◆ provsql_rmgr_desc()

void provsql_rmgr_desc ( StringInfo buf,
XLogReaderState * record )
static

Definition at line 117 of file provsql_rmgr.c.

◆ provsql_rmgr_identify()

const char * provsql_rmgr_identify ( uint8 info)
static

Definition at line 128 of file provsql_rmgr.c.

◆ provsql_rmgr_redo()

void provsql_rmgr_redo ( XLogReaderState * record)
static

Definition at line 88 of file provsql_rmgr.c.

Here is the call graph for this function:

◆ provsql_store_write_allowed()

bool provsql_store_write_allowed ( void )

Whether this process may write to the circuit store.

False in a hot-standby backend: the standby's store is maintained by replay, and a backend writing to it would make the two diverge for good. True in the startup process while it replays a ProvSQL record, which is the one write recovery is supposed to make.

Definition at line 157 of file provsql_rmgr.c.

Here is the caller graph for this function:

◆ provsql_wal_log_store_message()

void provsql_wal_log_store_message ( const char * data,
size_t len )

Write one store message to the WAL, if WAL logging is on.

Parameters
dataThe complete message, starting with its opcode byte.
lenIts length in bytes.

Definition at line 173 of file provsql_rmgr.c.

Here is the caller graph for this function:

Variable Documentation

◆ in_redo

bool in_redo = false
static

True while this process is replaying a ProvSQL WAL record, which is the one case where writing to the store during recovery is right.

Definition at line 64 of file provsql_rmgr.c.

◆ provsql_rmgr

const RmgrData provsql_rmgr
static
Initial value:
= {
.rm_name = "provsql",
.rm_redo = provsql_rmgr_redo,
.rm_desc = provsql_rmgr_desc,
.rm_identify = provsql_rmgr_identify,
.rm_startup = NULL,
.rm_cleanup = provsql_rmgr_cleanup,
.rm_mask = NULL,
.rm_decode = NULL,
}
static void provsql_rmgr_desc(StringInfo buf, XLogReaderState *record)
static void provsql_rmgr_cleanup(void)
End of WAL replay: the replay spool reaches the disk before the end-of-recovery checkpoint.
static void provsql_rmgr_redo(XLogReaderState *record)
static const char * provsql_rmgr_identify(uint8 info)

Definition at line 135 of file provsql_rmgr.c.

◆ provsql_store_wal_logging

bool provsql_store_wal_logging = false

Global variable set by the provsql.store_wal_logging run-time configuration parameter: when true, every mutation of the circuit store is written to the WAL before it is written to the store.

Definition at line 60 of file provsql_rmgr.c.