 |
ProvSQL C/C++ API
Adding support for provenance and uncertainty management to PostgreSQL databases
|
|
ProvSQL C/C++ API
Adding support for provenance and uncertainty management to PostgreSQL databases
|
Removing from the circuit store what nothing references any more. More...
#include "postgres.h"#include "access/htup_details.h"#include "catalog/pg_database.h"#include "commands/dbcommands.h"#include "executor/spi.h"#include "fmgr.h"#include "funcapi.h"#include "miscadmin.h"#include "storage/lmgr.h"#include "storage/procarray.h"#include "utils/builtins.h"#include "utils/guc.h"#include "utils/lsyscache.h"#include "utils/uuid.h"#include "circuit_cache.h"#include "provsql_mmap.h"#include "provsql_shmem.h"#include "provsql_utils.h"
Go to the source code of this file.
Classes | |
| struct | root_set |
| Growable array of root tokens gathered from the database. More... | |
Functions | |
| static void | root_set_add (root_set *rs, const pg_uuid_t *token) |
| static void | collect_from (root_set *rs, const char *query) |
Run query and append every UUID it returns to rs. | |
| static void | collect_roots (root_set *rs) |
| Gather every token stored in the database. | |
| Datum | circuit_cleanup (PG_FUNCTION_ARGS) |
| Rebuild this database's circuit store, keeping only what the tokens stored in the database reach. | |
Removing from the circuit store what nothing references any more.
The store only grows. A gate is never removed, and that is deliberate: gates are immutable and content-addressed, so a transaction that rolls back leaves an orphan rather than an inconsistency, and the same expression recomputed lands on the same gate. The price is that every dropped table, every remove_provenance, every reloaded dataset and every exploratory join over a large table leaves gates behind for good.
provsql.circuit_cleanup() is the complement of that: the one operation allowed to remove gates, run explicitly, the way VACUUM FULL is the complement of MVCC. It keeps every gate reachable from a token stored in the database and discards the rest, rewriting the four store files compactly on the way – which also makes it the repair tool for a store damaged by an interrupted write (see provsql.check_store).
Why it needs the database to itself. Two properties rule out a concurrent collector. Gates are re-created idempotently: a query running alongside can compute provenance_plus(a, b), find the content-addressed gate already there as an orphan, and reference it a moment before the collector – which saw no reference – removes it. And a transaction in progress has created gates and inserted rows that the collector's snapshot cannot see. So the function takes the database's lock in the mode DROP DATABASE takes it, and refuses to run while another session is connected, exactly as DROP DATABASE does.
What counts as a root. Every value of a uuid, agg_token or random_variable column – and of arrays of those – in every table and materialised view of the database, whatever the column is called: "CREATE TABLE ... AS SELECT provenance()" stores tokens under whatever name the user picks, mapping tables use provenance, update_provenance uses provsql, and users store get_children results and conditioning events wherever they like. Scanning by type is the only conservative choice, because a missed root turns a derived gate back into an input, silently. The semiring constants gate_zero and gate_one are roots unconditionally: the planner emits their UUIDs as literals, so nothing in the database need mention them.
A token that lives only outside the database – copied into a notebook cell or a deep link, kept in a file, stored as text or inside a jsonb document – is not a root. Content-addressed gates come back by re-running the query that produced them; freshly minted ones (an rv leaf, the update gate of a deleted log row, the input gate of a row deleted from an untracked copy) do not.
Definition in file circuit_cleanup.c.
| Datum circuit_cleanup | ( | PG_FUNCTION_ARGS | ) |
Rebuild this database's circuit store, keeping only what the tokens stored in the database reach.
Takes the database exclusively for the duration; see the file comment for why that is unavoidable. dry_run reports how much would be kept without writing anything.
Definition at line 228 of file circuit_cleanup.c.
|
static |
Run query and append every UUID it returns to rs.
The query is expected to return one uuid column.
Definition at line 94 of file circuit_cleanup.c.
|
static |
Gather every token stored in the database.
Walks pg_attribute for columns whose type is one ProvSQL tokens live in, then reads each of them. Provenance tracking is switched off for the duration: these reads would otherwise materialise an input gate per row of every tracked table, which is exactly the growth the clean-up is there to undo.
Definition at line 118 of file circuit_cleanup.c.
