ProvSQL C/C++ API
Adding support for provenance and uncertainty management to PostgreSQL databases
Loading...
Searching...
No Matches
provsql_mmap.h File Reference

Background worker and IPC primitives for mmap-backed circuit storage. More...

#include "limits.h"
#include <unistd.h>
#include "postgres.h"
#include "provsql_utils.h"
#include "provsql_config.h"
Include dependency graph for provsql_mmap.h:
This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes

struct  provsql_cleanup_result
 What provsql.circuit_cleanup() reports. More...

Macros

#define PROVSQL_STORE_FLUSH_INTERVAL_MS   200
 How long the worker waits after a write before forcing the store to stable storage.
#define PROVSQL_REPLAY_SPOOL   "global/provsql_replay_spool"
 Where WAL replay writes the store messages while the worker is not running, relative to the data directory.
#define READM(var, type)
 Read one value of type from the background-to-main pipe.
#define READB(var, type)
 Read one value of type from the main-to-background pipe.
#define WRITEB(pvar, type)
 Write one value of type to the main-to-background pipe.
#define WRITEM(pvar, type)
 Write one value of type to the background-to-main pipe.
#define READB_BYTES(ptr, n)
 Read exactly n bytes of a reply from the main-to-background pipe.
#define READM_BYTES(ptr, n)
 Read exactly n bytes of a request from the background-to-main pipe.
#define WRITEB_BYTES(ptr, n)
 Write n reply bytes to the main-to-background pipe.
#define STARTWRITEM()
 Reset the shared write buffer for a new batched write.
#define ADDWRITEM(pvar, type)
 Append one value of type to the shared write buffer.
#define SENDWRITEM()
 Flush the shared write buffer to the background-to-main pipe atomically.
#define ADDWRITEDB()
 Append the per-message database header to the write buffer.

Enumerations

enum  provsql_set_prob_result { PROVSQL_SET_PROB_NOT_PROB_GATE = 0 , PROVSQL_SET_PROB_WRITTEN = 1 , PROVSQL_SET_PROB_UNCHANGED = 2 , PROVSQL_SET_PROB_ALREADY_SET = 3 }
 Outcome of a probability write, mirroring MMappedCircuit::SetProbResult across the IPC boundary. More...

Functions

void provsql_mmap_worker (Datum)
 Entry point for the ProvSQL mmap background worker.
void RegisterProvSQLMMapWorker (void)
 Register the ProvSQL mmap background worker with PostgreSQL.
void initialize_provsql_mmap (void)
 Initialise the circuit store.
void destroy_provsql_mmap (void)
 Unmap and close the mmap files.
void provsql_mmap_main_loop (void)
 Main processing loop of the mmap background worker.
void provsql_circuit_cleanup_request (bool dry_run, const pg_uuid_t *roots, int64 nb_roots, provsql_cleanup_result *out)
 Ask the worker to rebuild this database's store, keeping only what roots reach.
void provsql_store_flush (void)
 Force every open circuit to stable storage.
void provsql_store_note_write (void)
 Note that this transaction has written to the circuit store.
bool provsql_store_written (void)
 Whether this transaction has written to the circuit store.
void provsql_mmap_dispatch (char c, Oid db_oid, Oid db_tablespace)
 Handle a single IPC message: read its payload and write its reply.
void provsql_internal_create_gate (const pg_uuid_t *token, gate_type type, unsigned nb_children, const pg_uuid_t *children_data)
 Create a gate from in-extension C/C++ code (cache + worker IPC).
void provsql_internal_create_gate_with (const pg_uuid_t *token, gate_type type, unsigned nb_children, const pg_uuid_t *children, bool has_infos, unsigned info1, unsigned info2, const char *extra)
 Create a gate together with its infos and its text, in one message that is not answered.
provsql_set_prob_result provsql_internal_set_prob (const pg_uuid_t *token, double prob, double *existing)
 Write a gate's probability from in-extension C/C++ code.
void provsql_internal_clear_prob (const pg_uuid_t *token)
 Drop a gate's probability, leaving it as it was before anyone wrote one.
bool provsql_internal_get_prob_written (const pg_uuid_t *token, double *prob)
 Report whether a probability has been written on a gate.
gate_type provsql_fetch_gate (const pg_uuid_t *token, unsigned *nb_children_out, pg_uuid_t **children_out)
 Fetch a gate's type and children, cache-first with a worker round-trip (and cache fill) on a miss.
void provsql_internal_get_infos (const pg_uuid_t *token, unsigned *info1, unsigned *info2)
 Read a gate's info1 and info2, through the worker.
char * provsql_internal_get_extra (const pg_uuid_t *token)
 Read a gate's extra text, through the worker; palloc'd.
bool provsql_read_all (int fd, void *dst, size_t n)
 Read exactly n bytes from fd into dst; false on EOF/error.
bool provsql_write_all (int fd, const void *src, size_t n)
 Write the n bytes of src to fd, however many write calls it takes; false on error.
void provsql_request_start (void)
 Start a request that may keep the worker busy (the building of a circuit): no cancel is pending for it yet.
bool provsql_wait_reply (void)
 Wait, under the store lock, until the worker's reply to this backend's request can be read.
bool provsql_worker_read (void *dst, size_t n)
 The worker's buffered read of the request pipe: what the pipe holds is read in one call, and the messages are parsed from the buffer.
bool provsql_worker_buffered (void)
 Whether the worker's read buffer holds unread bytes, which poll() cannot see.
bool provsql_worker_spooled (void)
 Whether store messages spooled during crash recovery remain to be applied (they come before the pipe).
void provsql_replay_spool_close (void)
 Sync and close the replay spool at the end of WAL replay (the resource manager's cleanup).
bool provsql_store_cancel_requested (void)
 Whether the circuit being built should be given up: in the worker, the backend that asked for it was cancelled (see provsql_wait_reply); in the single-process build, where the backend builds it itself, its statement was.

Variables

char buffer [PIPE_BUF]
 Shared write buffer used with STARTWRITEM / ADDWRITEM / SENDWRITEM.
unsigned bufferpos
 Current write position within buffer.
bool provsql_worker_from_spool
 Whether the message being applied comes from the replay spool: it is not answered, nobody waits for it.

Detailed Description

Background worker and IPC primitives for mmap-backed circuit storage.

ProvSQL persists the provenance circuit in memory-mapped files so that data survives transaction boundaries and is shared across backend processes. Because multiple backends may create gates concurrently, a dedicated PostgreSQL background worker (provsql_mmap_worker) is the sole writer to those files; normal backends communicate with it through a pair of anonymous pipes described in provsqlSharedState.

This header exposes:

  • Functions to register, start, and manage the background worker.
  • A set of pipe I/O macros (READM, READB, WRITEB, WRITEM) that wrap read()/write() calls on the inter-process pipes.
  • A buffered-write interface (STARTWRITEM, ADDWRITEM, SENDWRITEM) that batches multiple fields into a single write() to stay within the atomic PIPE_BUF guarantee.

Definition in file provsql_mmap.h.

Macro Definition Documentation

◆ ADDWRITEDB

#define ADDWRITEDB ( )
Value:
(ADDWRITEM(&MyDatabaseId, Oid), ADDWRITEM(&MyDatabaseTableSpace, Oid))
#define ADDWRITEM(pvar, type)
Append one value of type to the shared write buffer.

Append the per-message database header to the write buffer.

Every request carries the OID of the database it applies to and the OID of that database's default tablespace, so the worker – which runs outside any transaction and cannot read pg_database – can resolve the directory holding the backing files. Follows the opcode byte in every message.

Definition at line 373 of file provsql_mmap.h.

◆ ADDWRITEM

#define ADDWRITEM ( pvar,
type )
Value:
(memcpy(buffer+bufferpos, pvar, sizeof(type)), bufferpos+=sizeof(type))
char buffer[PIPE_BUF]
Shared write buffer used with STARTWRITEM / ADDWRITEM / SENDWRITEM.
unsigned bufferpos
Current write position within buffer.

Append one value of type to the shared write buffer.

Definition at line 358 of file provsql_mmap.h.

◆ PROVSQL_REPLAY_SPOOL

#define PROVSQL_REPLAY_SPOOL   "global/provsql_replay_spool"

Where WAL replay writes the store messages while the worker is not running, relative to the data directory.

Definition at line 334 of file provsql_mmap.h.

◆ PROVSQL_STORE_FLUSH_INTERVAL_MS

#define PROVSQL_STORE_FLUSH_INTERVAL_MS   200

How long the worker waits after a write before forcing the store to stable storage.

The circuit store is outside PostgreSQL's WAL, so a committed transaction's gates can still be sitting in the kernel's page cache when the machine loses power. Forcing them out this long after the last write bounds the loss, the way synchronous_commit = off bounds the heap's; provsql.store_synchronous_commit removes it entirely, at the price of one flush per store-writing transaction.

Definition at line 84 of file provsql_mmap.h.

◆ READB

#define READB ( var,
type )
Value:
provsql_read_all(provsql_shared_state->pipembr, &var, sizeof(type))
bool provsql_read_all(int fd, void *dst, size_t n)
Read exactly n bytes from fd into dst; false on EOF/error.
provsqlSharedState * provsql_shared_state
Pointer to the ProvSQL shared-memory segment (set in provsql_shmem_startup).

Read one value of type from the main-to-background pipe.

Definition at line 342 of file provsql_mmap.h.

◆ READB_BYTES

#define READB_BYTES ( ptr,
n )
Value:

Read exactly n bytes of a reply from the main-to-background pipe.

Definition at line 349 of file provsql_mmap.h.

◆ READM

#define READM ( var,
type )
Value:
provsql_worker_read(&var, sizeof(type))
bool provsql_worker_read(void *dst, size_t n)
The worker's buffered read of the request pipe: what the pipe holds is read in one call,...

Read one value of type from the background-to-main pipe.

Definition at line 340 of file provsql_mmap.h.

◆ READM_BYTES

#define READM_BYTES ( ptr,
n )
Value:

Read exactly n bytes of a request from the background-to-main pipe.

Definition at line 351 of file provsql_mmap.h.

◆ SENDWRITEM

#define SENDWRITEM ( )
Value:
bool provsql_write_all(int fd, const void *src, size_t n)
Write the n bytes of src to fd, however many write calls it takes; false on error.

Flush the shared write buffer to the background-to-main pipe atomically.

Definition at line 360 of file provsql_mmap.h.

◆ STARTWRITEM

#define STARTWRITEM ( )
Value:

Reset the shared write buffer for a new batched write.

Definition at line 356 of file provsql_mmap.h.

◆ WRITEB

#define WRITEB ( pvar,
type )
Value:
provsql_write_all(provsql_shared_state->pipembw, pvar, sizeof(type))

Write one value of type to the main-to-background pipe.

Definition at line 344 of file provsql_mmap.h.

◆ WRITEB_BYTES

#define WRITEB_BYTES ( ptr,
n )
Value:

Write n reply bytes to the main-to-background pipe.

Definition at line 353 of file provsql_mmap.h.

◆ WRITEM

#define WRITEM ( pvar,
type )
Value:
provsql_write_all(provsql_shared_state->pipebmw, pvar, sizeof(type))

Write one value of type to the background-to-main pipe.

Definition at line 346 of file provsql_mmap.h.

Enumeration Type Documentation

◆ provsql_set_prob_result

Outcome of a probability write, mirroring MMappedCircuit::SetProbResult across the IPC boundary.

Enumerator
PROVSQL_SET_PROB_NOT_PROB_GATE 

The gate carries no probability.

PROVSQL_SET_PROB_WRITTEN 

Written; undo on rollback.

PROVSQL_SET_PROB_UNCHANGED 

Already held exactly this value.

PROVSQL_SET_PROB_ALREADY_SET 

Holds a different value; refused.

Definition at line 174 of file provsql_mmap.h.

Function Documentation

◆ destroy_provsql_mmap()

void destroy_provsql_mmap ( void )

Unmap and close the mmap files.

Called by the background worker on shutdown to release resources and ensure all dirty pages are synced to disk via msync().

Definition at line 111 of file MMappedCircuit.cpp.

Here is the caller graph for this function:

◆ initialize_provsql_mmap()

void initialize_provsql_mmap ( void )

Initialise the circuit store.

Called once by the background worker at startup. The per-database files themselves are opened lazily, on the first message for their database.

Definition at line 106 of file MMappedCircuit.cpp.

Here is the caller graph for this function:

◆ provsql_circuit_cleanup_request()

void provsql_circuit_cleanup_request ( bool dry_run,
const pg_uuid_t * roots,
int64 nb_roots,
provsql_cleanup_result * out )

Ask the worker to rebuild this database's store, keeping only what roots reach.

The caller must hold the database exclusively; see provsql.circuit_cleanup in circuit_cleanup.c.

Definition at line 411 of file provsql_mmap.c.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ provsql_fetch_gate()

gate_type provsql_fetch_gate ( const pg_uuid_t * token,
unsigned * nb_children_out,
pg_uuid_t ** children_out )

Fetch a gate's type and children, cache-first with a worker round-trip (and cache fill) on a miss.

On return *children_out is a calloc'd array to be freed by the caller, or NULL when the gate has no children.

Fetch a gate's type and children, cache-first with a worker round-trip (and cache fill) on a miss.

On cache miss this fetches BOTH the gate type and its children from the worker, in one critical section, then caches them together. If we cached only the type (with an empty children list), a subsequent get_children() call for the same token would consult the cache, find the entry, and return 0 children : never querying the worker for the real children. provsql.provenance_evaluate hits exactly that pattern (it calls get_gate_type first, then unnest(get_children(...))) and silently folds plus/times gates over an empty set.

Fetch a gate's type and children, cache-first with a worker round-trip (and cache fill) on a miss. Factored out of the get_gate_type() wrapper for in-extension callers that walk the circuit from C (e.g. the annotation-transparent set_prob()). On return *children_out is a calloc'd array to be freed by the caller, or NULL when the gate has no children.

Definition at line 629 of file provsql_mmap.c.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ provsql_internal_clear_prob()

void provsql_internal_clear_prob ( const pg_uuid_t * token)

Drop a gate's probability, leaving it as it was before anyone wrote one.

The rollback path of probability_store.c, and the only way to unset a probability: there is none from SQL. Unlike a write it does not arm the at-commit sync barrier – it runs when the transaction that would have committed is already gone.

Definition at line 852 of file provsql_mmap.c.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ provsql_internal_create_gate()

void provsql_internal_create_gate ( const pg_uuid_t * token,
gate_type type,
unsigned nb_children,
const pg_uuid_t * children_data )

Create a gate from in-extension C/C++ code (cache + worker IPC).

Internal entry point behind the SQL-callable create_gate(), without Datum marshalling or gate-type-OID lookups; idempotent on already-mapped tokens.

Parameters
tokenUUID of the gate.
typeGate type.
nb_childrenNumber of children.
children_dataChild UUIDs (may be NULL when nb_children is 0).

Create a gate from in-extension C/C++ code (cache + worker IPC).

Factored out of the SQL-callable wrapper so in-extension C/C++ code (e.g. the decomposition-aligned reachability materialiser) can create gates without Datum marshalling or gate-type-OID lookups. Same semantics: write-through to the per-session cache, then the C message to the background worker; MMappedCircuit::createGate is idempotent on already-mapped tokens.

Definition at line 718 of file provsql_mmap.c.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ provsql_internal_create_gate_with()

void provsql_internal_create_gate_with ( const pg_uuid_t * token,
gate_type type,
unsigned nb_children,
const pg_uuid_t * children,
bool has_infos,
unsigned info1,
unsigned info2,
const char * extra )

Create a gate together with its infos and its text, in one message that is not answered.

For the gates whose address determines what they record (a value gate and its text, an annotation, a comparison and its operator, an aggregate), so that the write-once rule has nothing to refuse and nobody needs to wait for its answer. extra NULL records no text; has_infos false records no infos.

Definition at line 879 of file provsql_mmap.c.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ provsql_internal_get_extra()

char * provsql_internal_get_extra ( const pg_uuid_t * token)

Read a gate's extra text, through the worker; palloc'd.

Definition at line 1256 of file provsql_mmap.c.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ provsql_internal_get_infos()

void provsql_internal_get_infos ( const pg_uuid_t * token,
unsigned * info1,
unsigned * info2 )

Read a gate's info1 and info2, through the worker.

Definition at line 1238 of file provsql_mmap.c.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ provsql_internal_get_prob_written()

bool provsql_internal_get_prob_written ( const pg_uuid_t * token,
double * prob )

Report whether a probability has been written on a gate.

Distinct from get_prob(), which reports the value an evaluation would use and so answers 1 for a gate nobody gave a probability.

Parameters
tokenUUID of the gate.
probOn true return, the written probability.

Definition at line 857 of file provsql_mmap.c.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ provsql_internal_set_prob()

provsql_set_prob_result provsql_internal_set_prob ( const pg_uuid_t * token,
double prob,
double * existing )

Write a gate's probability from in-extension C/C++ code.

Probabilities are written once (see MMappedCircuit::setProb), so this reports which of the four cases applied rather than a bare success flag. Callers that write a probability on a gate they have just created can treat anything but PROVSQL_SET_PROB_NOT_PROB_GATE as success; set_prob() itself raises on PROVSQL_SET_PROB_ALREADY_SET.

Note that this is the raw store operation: it does not record the write in the transaction's undo list. SQL-level writers go through provsql_set_prob_tracked() in probability_store.c so a rollback drops what they wrote.

Parameters
tokenUUID of the gate.
probProbability value in [0,1], or NaN to clear.
existingOn PROVSQL_SET_PROB_ALREADY_SET, the stored value.

Definition at line 845 of file provsql_mmap.c.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ provsql_mmap_dispatch()

void provsql_mmap_dispatch ( char c,
Oid db_oid,
Oid db_tablespace )

Handle a single IPC message: read its payload and write its reply.

The opcode c and the message header (db_oid and the database's default tablespace db_tablespace) have already been consumed by the caller. Shared by the background-worker main loop (multi-process build) and the synchronous in-process dispatcher.

Handle a single IPC message: read its payload and write its reply.

A C++ exception escaping here – a file that cannot be opened or mapped, an allocation that fails – would reach the C frames of the worker and terminate it with an abort, which the postmaster answers by resetting every backend of the cluster. It is reported as the error it is instead, as the worker's other failures are: the worker exits and is restarted, and the other databases keep their sessions.

Definition at line 1031 of file MMappedCircuit.cpp.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ provsql_mmap_main_loop()

void provsql_mmap_main_loop ( void )

Main processing loop of the mmap background worker.

Waits for gate-creation requests from backend processes, processes them by writing to the mmap files, and handles SIGTERM for graceful shutdown.

Definition at line 1047 of file MMappedCircuit.cpp.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ provsql_mmap_worker()

void provsql_mmap_worker ( Datum ignored)

Entry point for the ProvSQL mmap background worker.

Called by the postmaster when it launches the background worker. Enters the main loop (provsql_mmap_main_loop()) and never returns normally. The single Datum argument is required by the background-worker API but is not used.

Definition at line 264 of file provsql_mmap.c.

Here is the call graph for this function:

◆ provsql_read_all()

bool provsql_read_all ( int fd,
void * dst,
size_t n )

Read exactly n bytes from fd into dst; false on EOF/error.

Definition at line 175 of file provsql_mmap.c.

Here is the caller graph for this function:

◆ provsql_replay_spool_close()

void provsql_replay_spool_close ( void )

Sync and close the replay spool at the end of WAL replay (the resource manager's cleanup).

Definition at line 470 of file provsql_mmap.c.

Here is the caller graph for this function:

◆ provsql_request_start()

void provsql_request_start ( void )

Start a request that may keep the worker busy (the building of a circuit): no cancel is pending for it yet.

Under the store lock.

Definition at line 212 of file provsql_mmap.c.

Here is the caller graph for this function:

◆ provsql_store_cancel_requested()

bool provsql_store_cancel_requested ( void )

Whether the circuit being built should be given up: in the worker, the backend that asked for it was cancelled (see provsql_wait_reply); in the single-process build, where the backend builds it itself, its statement was.

Definition at line 238 of file provsql_mmap.c.

Here is the caller graph for this function:

◆ provsql_store_flush()

void provsql_store_flush ( void )

Force every open circuit to stable storage.

Definition at line 568 of file MMappedCircuit.cpp.

Here is the caller graph for this function:

◆ provsql_store_note_write()

void provsql_store_note_write ( void )

Note that this transaction has written to the circuit store.

Arms the at-commit sync barrier (provsql.store_synchronous_commit) and the PREPARE TRANSACTION refusal.

Definition at line 397 of file provsql_mmap.c.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ provsql_store_written()

bool provsql_store_written ( void )

Whether this transaction has written to the circuit store.

Definition at line 406 of file provsql_mmap.c.

Here is the caller graph for this function:

◆ provsql_wait_reply()

bool provsql_wait_reply ( void )

Wait, under the store lock, until the worker's reply to this backend's request can be read.

A blocked read() would not see a cancel, a statement_timeout or a terminate: the signal handlers restart it, and interrupts are held with the lock anyway. The pipe is polled instead; on a cancel, the worker is asked to stop (provsql_store_cancel_requested), and the reply, which then comes soon, is still waited for: the caller reads it in full, so that the next request does not take it for its own, and only then, the lock released, gives up.

Returns
whether the statement was cancelled or the backend terminated.

Definition at line 217 of file provsql_mmap.c.

Here is the caller graph for this function:

◆ provsql_worker_buffered()

bool provsql_worker_buffered ( void )

Whether the worker's read buffer holds unread bytes, which poll() cannot see.

Definition at line 100 of file provsql_mmap.c.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ provsql_worker_read()

bool provsql_worker_read ( void * dst,
size_t n )

The worker's buffered read of the request pipe: what the pipe holds is read in one call, and the messages are parsed from the buffer.

false on EOF or error.

Definition at line 128 of file provsql_mmap.c.

◆ provsql_worker_spooled()

bool provsql_worker_spooled ( void )

Whether store messages spooled during crash recovery remain to be applied (they come before the pipe).

Definition at line 95 of file provsql_mmap.c.

Here is the caller graph for this function:

◆ provsql_write_all()

bool provsql_write_all ( int fd,
const void * src,
size_t n )

Write the n bytes of src to fd, however many write calls it takes; false on error.

Definition at line 193 of file provsql_mmap.c.

Here is the caller graph for this function:

◆ RegisterProvSQLMMapWorker()

void RegisterProvSQLMMapWorker ( void )

Register the ProvSQL mmap background worker with PostgreSQL.

Must be called from the extension's _PG_init() function so that the postmaster starts the worker on the next connection.

Definition at line 286 of file provsql_mmap.c.

Here is the caller graph for this function:

Variable Documentation

◆ buffer

char buffer[PIPE_BUF]
extern

Shared write buffer used with STARTWRITEM / ADDWRITEM / SENDWRITEM.

Definition at line 77 of file provsql_mmap.c.

◆ bufferpos

unsigned bufferpos
extern

Current write position within buffer.

Definition at line 78 of file provsql_mmap.c.

◆ provsql_worker_from_spool

bool provsql_worker_from_spool
extern

Whether the message being applied comes from the replay spool: it is not answered, nobody waits for it.

Definition at line 93 of file provsql_mmap.c.