MMappedCircuit.cpp File Reference

Persistent mmap-backed circuit: implementation and background-worker entry points. More...

#include <cerrno>
#include <cmath>
#include <map>
#include <sstream>
#include <string>
#include "MMappedCircuit.h"
#include "GenericCircuit.h"
#include "Circuit.hpp"
#include "provsql_utils_cpp.h"
#include <poll.h>
#include <fcntl.h>
#include <unistd.h>
#include <cstring>
#include "miscadmin.h"
#include "common/relpath.h"
#include "utils/palloc.h"
#include "provsql_mmap.h"
#include "provsql_shmem.h"

Include dependency graph for MMappedCircuit.cpp:

Go to the source code of this file.

Functions
void	initialize_provsql_mmap ()
	Initialise the circuit store.
void	destroy_provsql_mmap ()
	Unmap and close the mmap files.
static bool	carriesProb (gate_type type)
	Whether type is one of the gate kinds that carry a probability.
static void	finishInterruptedCleanup (Oid db_oid, Oid db_tablespace)
	Bring a database's store to a definite state before opening it.
static MMappedCircuit *	getCircuit (Oid db_oid, Oid db_tablespace)
	Return (creating lazily if needed) the circuit for db_oid.
static void	cleanupStore (Oid db_oid, Oid db_tablespace, const std::vector< pg_uuid_t > &roots, bool dry_run, provsql_cleanup_result *out)
	Rebuild a database's store, keeping only what roots reach.
void	provsql_store_flush (void)
	Force every open circuit to stable storage.
void	provsql_mmap_dispatch (char c, Oid db_oid, Oid db_tablespace)
	Handle a single IPC message: read its payload and write its reply.
void	provsql_mmap_main_loop ()
	Main processing loop of the mmap background worker.
bool	operator< (const pg_uuid_t a, const pg_uuid_t b)
	Lexicographic less-than comparison for pg_uuid_t.

Variables
static std::map< Oid, MMappedCircuit * >	circuits
	Per-database mmap-backed provenance circuits, keyed by database OID.
static bool	store_dirty = false
	Whether any circuit has been written to since the last flush.
static constexpr const char *	CLEANUP_SUFFIX = ".new"
	Suffix of the files a clean-up builds beside the live ones.
static constexpr const char *	CLEANUP_MARKER = "provsql_cleanup.commit"
	Marker file present only while a clean-up's rename sequence is in flight; see finishInterruptedCleanup.

Detailed Description

Persistent mmap-backed circuit: implementation and background-worker entry points.

Implements the MMappedCircuit methods declared in MMappedCircuit.h, the createGenericCircuit() free function, and the background-worker entry points declared in provsql_mmap.h:

• initialize_provsql_mmap(): called by the background worker at startup; the per-database MMappedCircuit instances themselves are opened lazily, on the first message for their database.
• destroy_provsql_mmap(): called on shutdown; syncs and deletes the singleton.
• provsql_mmap_main_loop(): the worker's main loop; receives gate- creation messages from backends over the IPC pipe and writes them to the mmap store.

The createGenericCircuit() function performs a BFS from a root UUID, reading gates from the mmap store and building an in-memory GenericCircuit.

Definition in file MMappedCircuit.cpp.

Function Documentation

carriesProb()

bool carriesProb ( gate_type type )

static

Whether type is one of the gate kinds that carry a probability.

Definition at line 173 of file MMappedCircuit.cpp.

Here is the caller graph for this function:

cleanupStore()

void cleanupStore ( Oid db_oid, Oid db_tablespace, const std::vector< pg_uuid_t > & roots, bool dry_run, provsql_cleanup_result * out )

static

Rebuild a database's store, keeping only what roots reach.

The store only ever grows: a gate is never removed, because removing one is unsafe while anything might still reference it, and because a content-addressed gate can be re-adopted by a query at any moment. This is the one operation allowed to remove gates, and it is safe only because the caller holds the database exclusively – no other session is connected, so nothing can adopt an orphan while the sweep runs.

On dry_run the mark phase runs and the counts are reported, but nothing is written.

Definition at line 410 of file MMappedCircuit.cpp.

Here is the call graph for this function:

Here is the caller graph for this function:

destroy_provsql_mmap()

void destroy_provsql_mmap

(

void

)

Unmap and close the mmap files.

Called by the background worker on shutdown to release resources and ensure all dirty pages are synced to disk via msync().

Definition at line 86 of file MMappedCircuit.cpp.

Here is the caller graph for this function:

finishInterruptedCleanup()

void finishInterruptedCleanup ( Oid db_oid, Oid db_tablespace )

static

Bring a database's store to a definite state before opening it.

A clean-up writes four new files, then renames them over the live ones one at a time. A crash inside that sequence would leave a mixture, so the sequence is bracketed by a marker file: present, it says the new files are complete and the renames must be finished; absent, it says any ".new" files are the debris of a clean-up that never got that far and are to be discarded.

Definition at line 362 of file MMappedCircuit.cpp.

Here is the call graph for this function:

Here is the caller graph for this function:

getCircuit()

MMappedCircuit * getCircuit ( Oid db_oid, Oid db_tablespace )

static

Return (creating lazily if needed) the circuit for db_oid.

Definition at line 386 of file MMappedCircuit.cpp.

Here is the call graph for this function:

Here is the caller graph for this function:

initialize_provsql_mmap()

void initialize_provsql_mmap

(

void

)

Initialise the circuit store.

Called once by the background worker at startup. The per-database files themselves are opened lazily, on the first message for their database.

Definition at line 81 of file MMappedCircuit.cpp.

Here is the caller graph for this function:

operator<()

bool operator<	(	const pg_uuid_t	a,
		const pg_uuid_t	b )

Lexicographic less-than comparison for pg_uuid_t.

Parameters

a	Left UUID.
b	Right UUID.

Returns

true if a is lexicographically less than b.

Definition at line 1088 of file MMappedCircuit.cpp.

provsql_mmap_dispatch()

void provsql_mmap_dispatch	(	char	c,
		Oid	db_oid,
		Oid	db_tablespace )

Handle a single IPC message: read its payload and write its reply.

The opcode c and the message header (db_oid and the database's default tablespace db_tablespace) have already been consumed by the caller. Shared by the background-worker main loop (multi-process build) and the synchronous in-process dispatcher.

Definition at line 499 of file MMappedCircuit.cpp.

Here is the call graph for this function:

Here is the caller graph for this function:

provsql_mmap_main_loop()

void provsql_mmap_main_loop

(

void

)

Main processing loop of the mmap background worker.

Waits for gate-creation requests from backend processes, processes them by writing to the mmap files, and handles SIGTERM for graceful shutdown.

Definition at line 860 of file MMappedCircuit.cpp.

Here is the call graph for this function:

Here is the caller graph for this function:

provsql_store_flush()

void provsql_store_flush

(

void

)

Force every open circuit to stable storage.

Definition at line 492 of file MMappedCircuit.cpp.

Here is the caller graph for this function:

Variable Documentation

circuits

std::map<Oid, MMappedCircuit*> circuits

static

Per-database mmap-backed provenance circuits, keyed by database OID.

Definition at line 46 of file MMappedCircuit.cpp.

CLEANUP_MARKER

const char* CLEANUP_MARKER = "provsql_cleanup.commit"

staticconstexpr

Marker file present only while a clean-up's rename sequence is in flight; see finishInterruptedCleanup.

Definition at line 350 of file MMappedCircuit.cpp.

CLEANUP_SUFFIX

const char* CLEANUP_SUFFIX = ".new"

staticconstexpr

Suffix of the files a clean-up builds beside the live ones.

Definition at line 347 of file MMappedCircuit.cpp.

store_dirty

bool store_dirty = false

static

Whether any circuit has been written to since the last flush.

The store's bytes reach the kernel as soon as a message is applied, but they reach the disk only when someone forces them; a machine that loses power in between loses whatever the kernel had not written back. The worker forces them a short while after the last write (see the main loop below), which bounds that loss the way synchronous_commit = off bounds the heap's.

Definition at line 56 of file MMappedCircuit.cpp.