Background worker registration and IPC primitives for mmap-backed storage. More...

#include "provsql_mmap.h"
#include "provsql_shmem.h"
#include "provsql_utils.h"
#include "MMappedTableInfo.h"
#include <errno.h>
#include <unistd.h>
#include <poll.h>
#include <math.h>
#include <assert.h>
#include "postgres.h"
#include "postmaster/bgworker.h"
#include "catalog/pg_type.h"
#include "fmgr.h"
#include "funcapi.h"
#include "utils/array.h"
#include "access/htup_details.h"
#include "utils/builtins.h"
#include "utils/inval.h"
#include "utils/syscache.h"
#include "circuit_cache.h"

Include dependency graph for provsql_mmap.c:

Go to the source code of this file.

Functions
void	provsql_mmap_worker (Datum ignored)
	Entry point for the ProvSQL mmap background worker.
void	RegisterProvSQLMMapWorker (void)
	Register the ProvSQL mmap background worker with PostgreSQL.
Datum	get_gate_type (PG_FUNCTION_ARGS)
	PostgreSQL-callable wrapper for get_gate_type().
Datum	create_gate (PG_FUNCTION_ARGS)
	PostgreSQL-callable wrapper for create_gate().
Datum	set_prob (PG_FUNCTION_ARGS)
	PostgreSQL-callable wrapper for set_prob().
Datum	set_infos (PG_FUNCTION_ARGS)
	PostgreSQL-callable wrapper for set_infos().
Datum	set_extra (PG_FUNCTION_ARGS)
	PostgreSQL-callable wrapper for set_extra().
Datum	get_extra (PG_FUNCTION_ARGS)
	PostgreSQL-callable wrapper for get_extra().
Datum	get_nb_gates (PG_FUNCTION_ARGS)
	PostgreSQL-callable wrapper for get_nb_gates().
Datum	get_children (PG_FUNCTION_ARGS)
	PostgreSQL-callable wrapper for get_children().
Datum	get_prob (PG_FUNCTION_ARGS)
	PostgreSQL-callable wrapper for get_prob().
static uint8_t	parse_table_kind (const char *label)
	Translate a SQL-side kind label into the persisted enum value.
static const char *	table_kind_label (uint8_t kind)
	Inverse of `parse_table_kind` for use by `get_table_info`.
Datum	set_table_info (PG_FUNCTION_ARGS)
	PostgreSQL-callable wrapper for setTableInfo() over the IPC pipe.
Datum	remove_table_info (PG_FUNCTION_ARGS)
	PostgreSQL-callable wrapper for removeTableInfo() over the IPC pipe.
bool	provsql_fetch_table_info (Oid relid, ProvenanceTableInfo *out)
	C-callable IPC fetch for per-table provenance metadata.
Datum	get_table_info (PG_FUNCTION_ARGS)
	PostgreSQL-callable wrapper around the cached table-info lookup.
Datum	set_ancestors (PG_FUNCTION_ARGS)
	PostgreSQL-callable wrapper for setTableAncestry() over the IPC pipe.
Datum	remove_ancestors (PG_FUNCTION_ARGS)
	PostgreSQL-callable wrapper for removeTableAncestry() over the IPC pipe.
bool	provsql_fetch_ancestry (Oid relid, uint16 ancestor_n_out, Oid ancestors_out)
	C-callable IPC fetch for the ancestor half of a per-table metadata record.
Datum	get_ancestors (PG_FUNCTION_ARGS)
	PostgreSQL-callable wrapper around the cached ancestry lookup.
Datum	get_infos (PG_FUNCTION_ARGS)
	PostgreSQL-callable wrapper for get_infos().

Variables
char	buffer [PIPE_BUF] ={}
	Shared write buffer used with `STARTWRITEM` / `ADDWRITEM` / `SENDWRITEM`.
unsigned	bufferpos =0
	Current write position within `buffer`.

Detailed Description

Background worker registration and IPC primitives for mmap-backed storage.

Implements the PostgreSQL background worker lifecycle functions declared in provsql_mmap.h:

RegisterProvSQLMMapWorker(): registers the worker with the postmaster during _PG_init().
provsql_mmap_worker(): worker entry point; sets up signal handlers and enters provsql_mmap_main_loop().

The IPC between normal backends and the background worker is handled in MMappedCircuit.cpp. This file provides the PostgreSQL-specific glue (background worker API, signal handling).

Also declares the shared write buffer buffer[] and position counter bufferpos used by the STARTWRITEM / ADDWRITEM / SENDWRITEM macros in provsql_mmap.h.

The gate-creation SQL functions (e.g. create_gate()) that backends call are also implemented here; they acquire the IPC lock, write a message to the background worker, and wait for an acknowledgment.

Definition in file provsql_mmap.c.

Function Documentation

◆ create_gate()

Datum create_gate ( PG_FUNCTION_ARGS )

PostgreSQL-callable wrapper for create_gate().

Definition at line 179 of file provsql_mmap.c.

Here is the call graph for this function:

◆ get_ancestors()

Datum get_ancestors ( PG_FUNCTION_ARGS )

PostgreSQL-callable wrapper around the cached ancestry lookup.

Returns NULL when no ancestor record exists (or the record is empty); otherwise an oid[] listing the base-relation OIDs. Goes through provsql_lookup_ancestry so repeated calls in the same session do not pay for IPC.

Definition at line 934 of file provsql_mmap.c.

Here is the call graph for this function:

◆ get_children()

Datum get_children ( PG_FUNCTION_ARGS )

PostgreSQL-callable wrapper for get_children().

Definition at line 426 of file provsql_mmap.c.

Here is the call graph for this function:

◆ get_extra()

Datum get_extra ( PG_FUNCTION_ARGS )

PostgreSQL-callable wrapper for get_extra().

Definition at line 368 of file provsql_mmap.c.

Here is the call graph for this function:

◆ get_gate_type()

Datum get_gate_type ( PG_FUNCTION_ARGS )

PostgreSQL-callable wrapper for get_gate_type().

On cache miss this fetches BOTH the gate type and its children from the worker, in one critical section, then caches them together. If we cached only the type (with an empty children list), a subsequent get_children() call for the same token would consult the cache, find the entry, and return 0 children : never querying the worker for the real children. provsql.provenance_evaluate hits exactly that pattern (it calls get_gate_type first, then unnest(get_children(...))) and silently folds plus/times gates over an empty set.

Definition at line 102 of file provsql_mmap.c.

Here is the call graph for this function:

◆ get_infos()

Datum get_infos ( PG_FUNCTION_ARGS )

PostgreSQL-callable wrapper for get_infos().

Definition at line 962 of file provsql_mmap.c.

Here is the call graph for this function:

◆ get_nb_gates()

Datum get_nb_gates ( PG_FUNCTION_ARGS )

PostgreSQL-callable wrapper for get_nb_gates().

Definition at line 404 of file provsql_mmap.c.

Here is the call graph for this function:

◆ get_prob()

Datum get_prob ( PG_FUNCTION_ARGS )

PostgreSQL-callable wrapper for get_prob().

Definition at line 503 of file provsql_mmap.c.

Here is the call graph for this function:

◆ get_table_info()

Datum get_table_info ( PG_FUNCTION_ARGS )

PostgreSQL-callable wrapper around the cached table-info lookup.

Returns NULL when no record exists for relid; otherwise a record (kind text, block_key int2[]) where kind is one of 'tid' / 'bid' / 'opaque'. Goes through provsql_lookup_table_info so repeated calls in the same session do not pay for IPC.

Definition at line 726 of file provsql_mmap.c.

Here is the call graph for this function:

◆ parse_table_kind()

uint8_t parse_table_kind ( const char * label )

static

Translate a SQL-side kind label into the persisted enum value.

Definition at line 532 of file provsql_mmap.c.

Here is the caller graph for this function:

◆ provsql_fetch_ancestry()

bool provsql_fetch_ancestry	(	Oid	relid,
		uint16 *	ancestor_n_out,
		Oid *	ancestors_out )

C-callable IPC fetch for the ancestor half of a per-table metadata record.

Raw IPC fetch for the ancestry half (no cache).

Sends an 'a' message to the background worker and reads back the response. No caching: every call hits the worker. Use provsql_lookup_ancestry for the cached, planner-hot-path variant.

Parameters

relid	pg_class OID of the relation to look up.
ancestor_n_out	On `true` return, count of valid entries.
ancestors_out	On `true` return, the ancestor OIDs (caller buffer of `PROVSQL_TABLE_INFO_MAX_ANCESTORS`).

Returns: true if the worker returned a non-zero ancestor count; false otherwise (no record, or empty ancestor set).

Definition at line 884 of file provsql_mmap.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ provsql_fetch_table_info()

bool provsql_fetch_table_info	(	Oid	relid,
		ProvenanceTableInfo *	out )

C-callable IPC fetch for per-table provenance metadata.

Raw IPC fetch (no cache).

Sends an 's' message to the background worker and reads back the response. No caching: every call hits the worker. Use provsql_lookup_table_info() for the cached, planner-hot-path variant.

Parameters

relid	pg_class OID of the relation to look up.
out	On success, filled with the stored record.

Returns: true if the worker returned a record, false otherwise.

Definition at line 680 of file provsql_mmap.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ provsql_mmap_worker()

void provsql_mmap_worker ( Datum ignored )

Entry point for the ProvSQL mmap background worker.

Called by the postmaster when it launches the background worker. Enters the main loop (provsql_mmap_main_loop()) and never returns normally. The single Datum argument is required by the background-worker API but is not used.

Definition at line 52 of file provsql_mmap.c.

Here is the call graph for this function:

◆ RegisterProvSQLMMapWorker()

void RegisterProvSQLMMapWorker ( void )

Register the ProvSQL mmap background worker with PostgreSQL.

Must be called from the extension's _PG_init() function so that the postmaster starts the worker on the next connection.

Definition at line 65 of file provsql_mmap.c.

Here is the caller graph for this function:

◆ remove_ancestors()

Datum remove_ancestors ( PG_FUNCTION_ARGS )

PostgreSQL-callable wrapper for removeTableAncestry() over the IPC pipe.

Clears just the ancestor half of a per-table metadata record, leaving kind / block_key intact. Use remove_table_info to delete the whole record instead.

Definition at line 840 of file provsql_mmap.c.

Here is the call graph for this function:

◆ remove_table_info()

Datum remove_table_info ( PG_FUNCTION_ARGS )

PostgreSQL-callable wrapper for removeTableInfo() over the IPC pipe.

Definition at line 638 of file provsql_mmap.c.

Here is the call graph for this function:

◆ set_ancestors()

Datum set_ancestors ( PG_FUNCTION_ARGS )

PostgreSQL-callable wrapper for setTableAncestry() over the IPC pipe.

Records the base-relation ancestor set of a tracked relation. relid is the pg_class OID of the relation; ancestors is an oid[] (possibly empty) listing the base add_provenance / repair_key relations this one's atoms ultimately come from. The worker preserves the relation's existing kind / block_key half on update.

Silently no-op on the worker side when relid has no kind record yet – the safe-query rewriter only consults ancestry for tracked relations, so callers should run add_provenance / repair_key / set_table_info first.

Definition at line 777 of file provsql_mmap.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ set_extra()

Datum set_extra ( PG_FUNCTION_ARGS )

PostgreSQL-callable wrapper for set_extra().

Definition at line 339 of file provsql_mmap.c.

Here is the call graph for this function:

◆ set_infos()

Datum set_infos ( PG_FUNCTION_ARGS )

PostgreSQL-callable wrapper for set_infos().

Definition at line 308 of file provsql_mmap.c.

Here is the call graph for this function:

◆ set_prob()

Datum set_prob ( PG_FUNCTION_ARGS )

PostgreSQL-callable wrapper for set_prob().

Definition at line 278 of file provsql_mmap.c.

Here is the call graph for this function:

◆ set_table_info()

Datum set_table_info ( PG_FUNCTION_ARGS )

PostgreSQL-callable wrapper for setTableInfo() over the IPC pipe.

Forward declaration of the C SQL entry points.

Stores per-relation provenance metadata used by the safe-query optimisation. relid is the pg_class OID of the relation; kind is one of the textual labels 'tid' / 'bid' / 'opaque' (see provsql_table_kind in MMappedTableInfo.h); block_key is an int2 array (possibly empty) listing the block-key column numbers when kind is 'bid'.

Definition at line 565 of file provsql_mmap.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ table_kind_label()

const char * table_kind_label ( uint8_t kind )

static

Inverse of parse_table_kind for use by get_table_info.

Definition at line 543 of file provsql_mmap.c.

Here is the caller graph for this function:

Variable Documentation

◆ buffer

char buffer[PIPE_BUF] ={}

Shared write buffer used with STARTWRITEM / ADDWRITEM / SENDWRITEM.

Definition at line 49 of file provsql_mmap.c.

◆ bufferpos

unsigned bufferpos =0

Current write position within buffer.

Definition at line 50 of file provsql_mmap.c.

Functions

Variables

Detailed Description

Function Documentation

◆ create_gate()

◆ get_ancestors()

◆ get_children()

◆ get_extra()

◆ get_gate_type()

◆ get_infos()

◆ get_nb_gates()

◆ get_prob()

◆ get_table_info()

◆ parse_table_kind()

◆ provsql_fetch_ancestry()

◆ provsql_fetch_table_info()

◆ provsql_mmap_worker()

◆ RegisterProvSQLMMapWorker()

◆ remove_ancestors()

◆ remove_table_info()

◆ set_ancestors()

◆ set_extra()

◆ set_infos()

◆ set_prob()

◆ set_table_info()

◆ table_kind_label()

Variable Documentation

◆ buffer

◆ bufferpos