ProvSQL C/C++ API
Adding support for provenance and uncertainty management to PostgreSQL databases
Loading...
Searching...
No Matches
MMappedCircuit.h
Go to the documentation of this file.
1/**
2 * @file MMappedCircuit.h
3 * @brief Persistent, mmap-backed storage for the full provenance circuit.
4 *
5 * @c MMappedCircuit is the authoritative store for all provenance circuit
6 * data that must survive transaction boundaries and be accessible across
7 * multiple PostgreSQL backends. It composes three @c MMappedVector
8 * instances plus one @c MMappedUUIDHashTable:
9 *
10 * | Component | Contents |
11 * |---------------------|-----------------------------------------------|
12 * | @c mapping | UUID → gate index (hash table) |
13 * | @c gates | @c GateInformation records, one per gate |
14 * | @c wires | Flattened child-UUID lists for all gates |
15 * | @c extra | Variable-length string data (e.g. provenance labels) |
16 *
17 * All four backing files live in the database's directory inside the
18 * PostgreSQL data directory, and are opened/created by the ProvSQL
19 * background worker on the first message for that database.
20 *
21 * The free-function @c createGenericCircuit() traverses the mmap data
22 * starting from a given root UUID to construct an in-memory
23 * @c GenericCircuit for evaluation.
24 *
25 * @warning ON-DISK ABI: the layouts of @c GateInformation, of the
26 * @c gate_type enum (defined in @c provsql_utils.h), of @c pg_uuid_t,
27 * and of @c MMappedUUIDHashTable's slot structure are all serialised
28 * verbatim into the four @c provsql_*.mmap backing files. ProvSQL
29 * supports in-place extension upgrades (@c ALTER @c EXTENSION @c provsql
30 * @c UPDATE) only because these layouts have been stable since
31 * ProvSQL 1.0.0. Any change that adds, removes, reorders, or resizes
32 * a field -- or that renumbers a @c gate_type enumerator -- silently
33 * breaks every existing installation's on-disk mmap files. If such a
34 * change is necessary, bump an explicit format-version header in the
35 * mmap files, write a migration path, and call it out in a release note.
36 */
37#ifndef MMAPPED_CIRCUIT_H
38#define MMAPPED_CIRCUIT_H
39
40#include <cmath>
41#include <cstddef>
42#include <cstdint>
43#include <cstring>
44#include <exception>
45#include <string>
46#include <vector>
47
48#include "GenericCircuit.h"
49#include "FormulaRegions.h"
51#include "MMappedVector.hpp"
52
53extern "C" {
54#include "provsql_utils.h"
55}
56
57/**
58 * @brief Per-gate metadata stored in the @c gates @c MMappedVector.
59 *
60 * Each gate in the persistent circuit has exactly one @c GateInformation
61 * record. The @c children_idx and @c nb_children fields together index
62 * into the @c wires @c MMappedVector to find the gate's children.
63 * Similarly, @c extra_idx and @c extra_len index into the @c extra vector
64 * for variable-length string annotations.
65 */
66typedef struct GateInformation
67{
68 gate_type type; ///< Kind of gate (input, plus, times…)
69 unsigned nb_children; ///< Number of children
70 unsigned long children_idx;///< Start index of this gate's children in @c wires
71 double prob; ///< Associated probability, @c NaN when unset
72 unsigned info1; ///< General-purpose integer annotation 1
73 unsigned info2; ///< General-purpose integer annotation 2
74 unsigned long extra_idx; ///< Start index in @c extra for string data
75 unsigned extra_len; ///< Byte length of the string data in @c extra
76
77 /**
78 * @brief Construct a @c GateInformation with mandatory fields.
79 * @param t Gate type.
80 * @param n Number of children.
81 * @param i Start index of children in the @c wires vector.
82 */
83 GateInformation(gate_type t, unsigned n, unsigned long i) :
84 type(t), nb_children(n), children_idx(i), prob(NAN), info1(0), info2(0), extra_idx(0), extra_len(0) {
85 }
87
88/**
89 * @brief Persistent mmap-backed representation of the provenance circuit.
90 *
91 * @c MMappedCircuit is the single writer for circuit data; only the
92 * background worker should call its mutating methods. Reading methods
93 * may be called from any process that has mapped the files read-only.
94 */
96private:
97MMappedUUIDHashTable mapping; ///< UUID → gate-index hash table
98MMappedVector<GateInformation> gates; ///< Gate metadata array
99MMappedVector<pg_uuid_t> wires; ///< Flattened child UUID array
100MMappedVector<char> extra; ///< Variable-length string data
101
102static constexpr const char *GATES_FILENAME="provsql_gates.mmap"; ///< Backing file for @c gates
103static constexpr const char *WIRES_FILENAME="provsql_wires.mmap"; ///< Backing file for @c wires
104static constexpr const char *MAPPING_FILENAME="provsql_mapping.mmap"; ///< Backing file for @c mapping
105static constexpr const char *EXTRA_FILENAME="provsql_extra.mmap"; ///< Backing file for @c extra
106
107/** @brief Append a complete gate record, then publish @p token for it. */
108void appendGate(pg_uuid_t token, gate_type type,
109 const std::vector<pg_uuid_t> &children);
110
111/** @brief Whether the gate record at index @p idx holds a written
112 * probability (see @c GATES_VERSION for the version-1 leniency). */
113bool hasProbAt(unsigned long idx) const;
114
115
116
117/** @brief Delegating constructor that accepts pre-built paths. */
118MMappedCircuit(const std::string &mp, const std::string &gp,
119 const std::string &wp, const std::string &ep,
120 bool read_only) :
121 mapping(mp.c_str(), read_only, MAGIC_MAPPING),
122 gates (gp.c_str(), read_only, MAGIC_GATES, GATES_VERSION),
123 wires (wp.c_str(), read_only, MAGIC_WIRES),
124 extra (ep.c_str(), read_only, MAGIC_EXTRA) {}
125
126public:
127/**
128 * @brief Format version of the @c gates file this build writes.
129 *
130 * Version 1 stored @c 1.0 in @c GateInformation::prob both for a gate
131 * whose probability had been set to 1 and for one that had never been
132 * given a probability at all. Version 2 writes @c NaN for the latter,
133 * which is what lets a probability be written once and refused
134 * afterwards. A version-1 file is still read: @c 1.0 on a
135 * probability-bearing gate is then treated as unset, so a store
136 * carried across an upgrade keeps accepting the probabilities its
137 * owner has always been allowed to write. The ambiguity lasts until
138 * @c circuit_cleanup rewrites the file, which normalises those
139 * @c 1.0 values and stamps the file version 2.
140 */
141static constexpr uint16_t GATES_VERSION = 2;
142
143/**
144 * @brief Outcome of @c setProb.
145 *
146 * @c Written and @c Unchanged are both successes; only @c Written
147 * needs undoing if the writing transaction rolls back.
148 */
149enum class SetProbResult {
150 NotProbGate, ///< The token names a gate that carries no probability
151 Written, ///< The gate had no probability and now holds the given one
152 Unchanged, ///< The gate already held exactly this probability
153 AlreadySet ///< The gate holds a different probability; refused
154};
155
156
157/** @brief Build the full path of a file in a database's store directory,
158 * the one @c GetDatabasePath resolves for @p db_oid in @p db_tablespace. */
159static std::string storePath(Oid db_oid, Oid db_tablespace,
160 const char *filename);
161
162/** @brief Whether the store of @p db_oid in @p db_tablespace still has its
163 * gates file: false once the database is dropped. */
164static bool storeExists(Oid db_oid, Oid db_tablespace);
165
166/**
167 * @brief Outcome of @c setInfos or @c setExtra.
168 *
169 * Same discipline as @c setProb: an annotation is a fact appended to the
170 * gate, written once and idempotent on the same value.
171 */
173 NoSuchGate, ///< The token names no gate
174 Written, ///< The gate had none and now holds the given annotation
175 Unchanged, ///< The gate already held exactly this annotation
176 AlreadySet ///< The gate holds a different annotation; refused
177};
178
179/** @brief 8-byte magic constants identifying each mmap file type. */
180static constexpr uint64_t MAGIC_GATES =
181 uint64_t('P') | uint64_t('v') << 8 | uint64_t('S') << 16 | uint64_t('G') << 24 |
182 uint64_t('a') << 32 | uint64_t('t') << 40 | uint64_t('e') << 48 | uint64_t('s') << 56;
183static constexpr uint64_t MAGIC_WIRES =
184 uint64_t('P') | uint64_t('v') << 8 | uint64_t('S') << 16 | uint64_t('W') << 24 |
185 uint64_t('i') << 32 | uint64_t('r') << 40 | uint64_t('e') << 48 | uint64_t('s') << 56;
186static constexpr uint64_t MAGIC_MAPPING =
187 uint64_t('P') | uint64_t('v') << 8 | uint64_t('S') << 16 | uint64_t('M') << 24 |
188 uint64_t('a') << 32 | uint64_t('p') << 40 | uint64_t('n') << 48 | uint64_t('g') << 56;
189static constexpr uint64_t MAGIC_EXTRA =
190 uint64_t('P') | uint64_t('v') << 8 | uint64_t('S') << 16 | uint64_t('E') << 24 |
191 uint64_t('x') << 32 | uint64_t('t') << 40 | uint64_t('r') << 48 | uint64_t('a') << 56;
192
193/**
194 * @brief Open all four mmap backing files for the given database.
195 * @param db_oid OID of the target database.
196 * @param db_tablespace OID of that database's default tablespace; the
197 * files go in the directory @c GetDatabasePath
198 * resolves for the pair.
199 * @param read_only If @c true, all files are mapped read-only.
200 */
201MMappedCircuit(Oid db_oid, Oid db_tablespace, bool read_only = false);
202
203/** @brief Sync all backing files before destruction. */
205 sync();
206}
207
208/**
209 * @brief Persist a new gate to the mmap store.
210 *
211 * Allocates a @c GateInformation record, appends the children to the
212 * @c wires vector, and records the UUID→index mapping. Existing gates
213 * with the same @p token are silently skipped.
214 *
215 * @param token UUID identifying the new gate.
216 * @param type Gate type.
217 * @param children Ordered list of child gate UUIDs.
218 */
219void createGate(pg_uuid_t token, gate_type type, const std::vector<pg_uuid_t> &children);
220
221/**
222 * @brief Write the @c info1 / @c info2 annotations of a gate, once.
223 *
224 * The two fields are written once **each**, with @c 0 meaning "nothing
225 * recorded": a field goes from @c 0 to a value once, accepts that value
226 * again, and refuses a different one; writing @c 0 over a value records
227 * nothing and leaves it alone. Per field rather than per pair because
228 * the two are written by different parties -- a certified @c plus gate
229 * is marked as certified when it is built and tagged with the route that
230 * made it a query's root afterwards.
231 *
232 * @param token UUID of the gate to annotate.
233 * @param info1 Value for @c info1, or @c 0 to leave it alone.
234 * @param info2 Value for @c info2, or @c 0 to leave it alone.
235 * @param existing On @c AlreadySet, the pair the gate holds.
236 */
237SetAnnotationResult setInfos(pg_uuid_t token, unsigned info1, unsigned info2,
238 std::pair<unsigned, unsigned> *existing = nullptr);
239
240/**
241 * @brief Attach a variable-length string annotation to a gate, once.
242 *
243 * A gate with no annotation has an empty one, and an empty string is not
244 * an annotation, so "nothing recorded" and "recorded as nothing"
245 * coincide and the question of an unset marker does not arise. Offering
246 * the bytes the gate already holds is a no-op rather than a fresh
247 * append, which is what keeps the @c extra file from accumulating
248 * abandoned copies of the same string.
249 *
250 * @param token UUID of the gate.
251 * @param s String to store.
252 * @param existing On @c AlreadySet, the string the gate holds.
253 */
254SetAnnotationResult setExtra(pg_uuid_t token, const std::string &s,
255 std::string *existing = nullptr);
256
257/**
258 * @brief Write a gate's probability, once.
259 *
260 * A probability is a fact appended to the circuit, like the gate
261 * itself: it can be written when the gate has none, re-written with
262 * the identical value (so setup scripts and notebook cells stay
263 * re-runnable), and otherwise refused. Changing one means minting a
264 * fresh input gate and rewriting the rows that carry the old token --
265 * see @c provsql.replace_input.
266 *
267 * Passing @c NaN clears the probability unconditionally; that is how a
268 * transaction that rolls back drops the probabilities it wrote, and it
269 * is not reachable from SQL.
270 *
271 * If the token is not yet in the circuit, an input gate is created
272 * lazily.
273 *
274 * @param token UUID of the gate.
275 * @param prob Probability value in [0, 1], or @c NaN to clear.
276 * @param existing On @c AlreadySet, the probability the gate holds.
277 * @return Which of the four cases applied.
278 */
279SetProbResult setProb(pg_uuid_t token, double prob, double *existing = nullptr);
280
281/**
282 * @brief Report whether @p token names a gate that holds a probability.
283 * @param token UUID of the gate.
284 * @param prob On @c true return, the stored probability.
285 * @return @c true when the gate exists, carries probabilities, and one
286 * has been written to it.
287 */
288bool hasProb(pg_uuid_t token, double *prob) const;
289
290/**
291 * @brief Flush all backing files to disk with @c msync().
292 */
293void sync();
294
295/**
296 * @brief Force every backing file to stable storage.
297 *
298 * @c sync() pushes the regions' bytes into the files; this pushes the
299 * files out of the kernel's page cache, which is what a crash of the
300 * machine can otherwise lose.
301 */
302void flush();
303
304/**
305 * @brief Whether any backing file was found still marked open-for-writing
306 * when it was opened -- the previous writer died mid-write.
307 */
308bool uncleanShutdown() const;
309
310/**
311 * @brief What @c provsql.check_store() reports about a store.
312 *
313 * Every count but @c unclean is zero for a store nothing has damaged.
314 * A non-zero one means a write was interrupted at a point the ordering
315 * rules do not cover, or a set of files was copied at different instants
316 * (a file-level backup of a running server); @c provsql.circuit_cleanup()
317 * rebuilds the store from what is still reachable.
318 */
319struct Check {
320 bool unclean; ///< A file was left marked open-for-writing
321 unsigned long nb_gates; ///< Gate records
322 unsigned long nb_mapping; ///< Mapping entries
323 unsigned long next_value; ///< Next index the mapping would assign
324 unsigned long dangling_indices; ///< Mapping entries indexing past the records
325 unsigned long unreferenced; ///< Records no mapping entry points at
326 unsigned long bad_wires; ///< Records whose children run past the wires
327 unsigned long bad_extra; ///< Records whose extra runs past the extra file
328};
329
330/** @brief Walk the store and report what does not add up. */
331Check check() const;
332
333/**
334 * @brief Mark every gate reachable from @p roots.
335 *
336 * @param roots Token UUIDs to start from; unknown ones are ignored.
337 * @param live Filled with one flag per gate record.
338 * @return The number of live gates.
339 */
340unsigned long mark(const std::vector<pg_uuid_t> &roots,
341 std::vector<bool> &live) const;
342
343/** @brief The size of a store, in the three units that matter. */
344struct Counts {
345 unsigned long gates; ///< Gate records
346 unsigned long wires; ///< Child wires
347 unsigned long extra_bytes; ///< Bytes of variable-length annotation
348};
349
350/** @brief This store's current size. */
351inline Counts counts() const {
352 return { gates.nbElements(), wires.nbElements(), extra.nbElements() };
353}
354
355/**
356 * @brief Copy the gates flagged in @p live into a fresh set of files
357 * beside the current ones (suffix @c ".new").
358 *
359 * Child indices, extra offsets and the token table are all rebuilt, so
360 * the result is a compact, freshly hashed store. A version-1 @c gates
361 * file also gets its probabilities normalised on the way through: the
362 * @c 1.0 that used to mean both "written as certain" and "never written"
363 * becomes @c NaN on gates the file cannot prove were written, and the new
364 * file is stamped version 2, after which the ambiguity is gone.
365 *
366 * @return The gate, wire and extra-byte counts of the new files.
367 */
368Counts sweepInto(const std::vector<bool> &live,
369 const std::string &mp, const std::string &gp,
370 const std::string &wp, const std::string &ep) const;
371
372/** @brief Whether this store's @c gates file predates the @c NaN
373 * unset-probability convention (see @c GATES_VERSION). */
374inline bool legacyProbabilities() const {
375 return gates.version() < GATES_VERSION;
376}
377
378
379
380/**
381 * @brief Return the type of the gate identified by @p token.
382 * @param token UUID of the gate.
383 * @return The gate's type, or @c gate_input if not found (lazy default).
384 */
385gate_type getGateType(pg_uuid_t token) const;
386
387/**
388 * @brief Return the child UUIDs of the gate identified by @p token.
389 * @param token UUID of the gate.
390 * @return Ordered vector of child UUIDs.
391 */
392std::vector<pg_uuid_t> getChildren(pg_uuid_t token) const;
393
394/**
395 * @brief Return the probability an evaluation would use for @p token.
396 *
397 * A gate nobody has given a probability evaluates as certain, and a
398 * repaired row nobody has given one evaluates at the uniform weight of
399 * its @c repair_key block; either way this answers a number, not "unset".
400 * Use @c hasProb to tell the two apart.
401 *
402 * @param token UUID of the gate.
403 * @return The probability; @c NaN when @p token names no gate, or
404 * one that carries no probability at all.
405 */
406double getProb(pg_uuid_t token) const;
407
408/**
409 * @brief Return the @c info1 / @c info2 pair for the gate @p token.
410 * @param token UUID of the gate.
411 * @return Pair @c {info1, info2}, or @c {0,0} if not found.
412 */
413std::pair<unsigned, unsigned> getInfos(pg_uuid_t token) const;
414
415/**
416 * @brief Return the variable-length string annotation for gate @p token.
417 * @param token UUID of the gate.
418 * @return The stored string, or empty if none.
419 */
420std::string getExtra(pg_uuid_t token) const;
421
422/**
423 * @brief Return the total number of gates stored in the circuit.
424 * @return Total gate count.
425 */
426inline unsigned long getNbGates() const {
427 return gates.nbElements();
428}
429
430/**
431 * @brief Build an in-memory @c GenericCircuit rooted at @p token.
432 *
433 * Performs a depth-first traversal of the mmap-backed circuit starting
434 * from @p token and copies all reachable gates and wires into a newly
435 * constructed @c GenericCircuit.
436 *
437 * @param token UUID of the root gate.
438 * @param budget Bytes the process's resident memory may grow by while the
439 * circuit is built, 0 for no limit; past it,
440 * @c CircuitBudgetExceeded is thrown. On a cancel
441 * (@c provsql_store_cancel_requested), @c CircuitCancelled is.
442 * @return An in-memory @c GenericCircuit containing the sub-circuit.
443 */
445 std::size_t budget = 0) const;
446
447/**
448 * @brief Build an in-memory @c GenericCircuit reachable from any of
449 * @p roots.
450 *
451 * Multi-root variant of @c createGenericCircuit. Seeds the BFS with
452 * every UUID in @p roots so a shared subgraph reachable from more
453 * than one root is represented by a single @c gate_t (the
454 * @c GenericCircuit::setGate / @c getGate pair is idempotent on the
455 * UUID key). Used by @c getJointCircuit to load an RV's sub-DAG
456 * together with a conditioning gate that sits above it in the
457 * persisted DAG.
458 *
459 * @param roots UUIDs whose reachable closure to load. Order is
460 * irrelevant; identical UUIDs collapse via the
461 * @c std::set deduplication of the work list.
462 * @param budget As for the single-root variant.
463 * @return An in-memory @c GenericCircuit containing every gate
464 * reachable from any root.
465 */
467 const std::vector<pg_uuid_t> &roots, std::size_t budget = 0) const;
468
469/**
470 * @brief The regions of @p requests, for @c sr_formula_rows (see
471 * @c FormulaRegions).
472 *
473 * @param requests The gates, or ranges of their children, asked for.
474 * @param cap Size, in symbols, past which a gate is named.
475 * @param wide_cap Size past which a named gate's row is split into one
476 * row per child.
477 */
478FormulaRegions formulaRegions(const std::vector<FormulaRequest> &requests,
479 unsigned long cap,
480 unsigned long wide_cap) const;
481
482private:
483/** @brief Add the gate @p uuid to @p result, with what it records but its
484 * wires. */
485gate_t copyGate(GenericCircuit &result, pg_uuid_t uuid) const;
486/** @brief Give the gate @p id of @p result what the record @p idx (@c
487 * NOTHING for a gate not stored: an input) records but its wires. */
488void copyGateAt(GenericCircuit &result, gate_t id, unsigned long idx) const;
489/** @brief @c getProb of the record @p idx. */
490double probAt(unsigned long idx) const;
491/** @brief @c getExtra of the record @p idx. */
492std::string extraAt(unsigned long idx) const;
493};
494
495/**
496 * @brief Thrown by @c MMappedCircuit::createGenericCircuit when the circuit
497 * it builds takes the process past the memory budget it was given
498 * (@c provsql.max_memory, what the statement has not used of it).
499 */
500class CircuitBudgetExceeded : public std::exception {
501public:
502 const char *what() const noexcept override {
503 return "the circuit is larger than provsql.max_memory allows";
504 }
505};
506
507/** @brief The size the worker answers a @c 'g' / @c 'j' request with, in
508 * place of a circuit, when building it went past the budget the request
509 * carried. */
510constexpr unsigned long PROVSQL_CIRCUIT_OVER_BUDGET = ~0UL;
511
512/**
513 * @brief Thrown by @c MMappedCircuit::createGenericCircuit when the
514 * statement that asked for the circuit is cancelled
515 * (@c provsql_store_cancel_requested).
516 */
517class CircuitCancelled : public std::exception {
518public:
519 const char *what() const noexcept override {
520 return "the statement was cancelled";
521 }
522};
523
524/** @brief The size the worker answers a @c 'g' / @c 'j' request with, in
525 * place of a circuit, when the requesting statement was cancelled. */
526constexpr unsigned long PROVSQL_CIRCUIT_CANCELLED = ~0UL - 1;
527
528#ifdef PROVSQL_INPROCESS_STORE
529/**
530 * @brief Build a @c GenericCircuit rooted at @p token directly from the
531 * current backend's store.
532 *
533 * Single-process replacement for the @c 'g' IPC round-trip: there is no
534 * worker/backend boundary to ship a Boost-serialised copy across, so the
535 * backend constructs the circuit in place.
536 */
537GenericCircuit provsql_inproc_generic_circuit(pg_uuid_t token,
538 std::size_t budget);
539/** @brief Multi-root variant (single-process replacement for @c 'j'). */
540GenericCircuit provsql_inproc_joint_circuit(
541 const std::vector<pg_uuid_t> &roots, std::size_t budget);
542/** @brief Single-process replacement for @c 'F'. */
543FormulaRegions provsql_inproc_formula_regions(
544 const std::vector<FormulaRequest> &requests, unsigned long cap,
545 unsigned long wide_cap);
546#endif
547
548
549#endif /* MMAPPED_CIRCUIT_H */
gate_t
Strongly-typed gate identifier.
Definition Circuit.h:53
The parts of a circuit sr_formula_rows renders a few rows from.
Semiring-agnostic in-memory provenance circuit.
constexpr unsigned long PROVSQL_CIRCUIT_OVER_BUDGET
The size the worker answers a 'g' / 'j' request with, in place of a circuit, when building it went pa...
constexpr unsigned long PROVSQL_CIRCUIT_CANCELLED
The size the worker answers a 'g' / 'j' request with, in place of a circuit, when the requesting stat...
Open-addressing hash table mapping UUIDs to integers, backed by an mmap file.
Template implementation of MMappedVector<T>.
Thrown by MMappedCircuit::createGenericCircuit when the circuit it builds takes the process past the ...
const char * what() const noexcept override
Thrown by MMappedCircuit::createGenericCircuit when the statement that asked for the circuit is cance...
const char * what() const noexcept override
static constexpr uint16_t GATES_VERSION
Format version of the gates file this build writes.
unsigned long mark(const std::vector< pg_uuid_t > &roots, std::vector< bool > &live) const
Mark every gate reachable from roots.
FormulaRegions formulaRegions(const std::vector< FormulaRequest > &requests, unsigned long cap, unsigned long wide_cap) const
The regions of requests, for sr_formula_rows (see FormulaRegions).
SetAnnotationResult setInfos(pg_uuid_t token, unsigned info1, unsigned info2, std::pair< unsigned, unsigned > *existing=nullptr)
Write the info1 / info2 annotations of a gate, once.
MMappedUUIDHashTable mapping
UUID → gate-index hash table.
void createGate(pg_uuid_t token, gate_type type, const std::vector< pg_uuid_t > &children)
Persist a new gate to the mmap store.
Counts counts() const
This store's current size.
std::string getExtra(pg_uuid_t token) const
Return the variable-length string annotation for gate token.
bool legacyProbabilities() const
Whether this store's gates file predates the NaN unset-probability convention (see GATES_VERSION).
SetProbResult setProb(pg_uuid_t token, double prob, double *existing=nullptr)
Write a gate's probability, once.
void appendGate(pg_uuid_t token, gate_type type, const std::vector< pg_uuid_t > &children)
Append a complete gate record, then publish token for it.
double probAt(unsigned long idx) const
getProb of the record idx.
unsigned long getNbGates() const
Return the total number of gates stored in the circuit.
SetProbResult
Outcome of setProb.
@ NotProbGate
The token names a gate that carries no probability.
@ Unchanged
The gate already held exactly this probability.
@ Written
The gate had no probability and now holds the given one.
@ AlreadySet
The gate holds a different probability; refused.
std::string extraAt(unsigned long idx) const
getExtra of the record idx.
static constexpr const char * GATES_FILENAME
Backing file for gates.
Counts sweepInto(const std::vector< bool > &live, const std::string &mp, const std::string &gp, const std::string &wp, const std::string &ep) const
Copy the gates flagged in live into a fresh set of files beside the current ones (suffix "....
gate_type getGateType(pg_uuid_t token) const
Return the type of the gate identified by token.
Check check() const
Walk the store and report what does not add up.
void sync()
Flush all backing files to disk with msync().
static std::string storePath(Oid db_oid, Oid db_tablespace, const char *filename)
Build the full path of a file in a database's store directory, the one GetDatabasePath resolves for d...
static constexpr const char * WIRES_FILENAME
Backing file for wires.
void copyGateAt(GenericCircuit &result, gate_t id, unsigned long idx) const
Give the gate id of result what the record idx (NOTHING for a gate not stored: an input) records but ...
MMappedCircuit(const std::string &mp, const std::string &gp, const std::string &wp, const std::string &ep, bool read_only)
Delegating constructor that accepts pre-built paths.
gate_t copyGate(GenericCircuit &result, pg_uuid_t uuid) const
Add the gate uuid to result, with what it records but its wires.
static bool storeExists(Oid db_oid, Oid db_tablespace)
Whether the store of db_oid in db_tablespace still has its gates file: false once the database is dro...
bool uncleanShutdown() const
Whether any backing file was found still marked open-for-writing when it was opened – the previous wr...
static constexpr const char * EXTRA_FILENAME
Backing file for extra.
static constexpr uint64_t MAGIC_WIRES
static constexpr uint64_t MAGIC_GATES
8-byte magic constants identifying each mmap file type.
static constexpr const char * MAPPING_FILENAME
Backing file for mapping.
bool hasProb(pg_uuid_t token, double *prob) const
Report whether token names a gate that holds a probability.
MMappedVector< char > extra
Variable-length string data.
static constexpr uint64_t MAGIC_EXTRA
double getProb(pg_uuid_t token) const
Return the probability an evaluation would use for token.
std::vector< pg_uuid_t > getChildren(pg_uuid_t token) const
Return the child UUIDs of the gate identified by token.
MMappedVector< GateInformation > gates
Gate metadata array.
SetAnnotationResult setExtra(pg_uuid_t token, const std::string &s, std::string *existing=nullptr)
Attach a variable-length string annotation to a gate, once.
SetAnnotationResult
Outcome of setInfos or setExtra.
@ NoSuchGate
The token names no gate.
~MMappedCircuit()
Sync all backing files before destruction.
bool hasProbAt(unsigned long idx) const
Whether the gate record at index idx holds a written probability (see GATES_VERSION for the version-1...
MMappedVector< pg_uuid_t > wires
Flattened child UUID array.
GenericCircuit createGenericCircuit(pg_uuid_t token, std::size_t budget=0) const
Build an in-memory GenericCircuit rooted at token.
static constexpr uint64_t MAGIC_MAPPING
std::pair< unsigned, unsigned > getInfos(pg_uuid_t token) const
Return the info1 / info2 pair for the gate token.
void flush()
Force every backing file to stable storage.
Persistent open-addressing hash table mapping UUIDs to integers.
Append-only, mmap-backed vector of elements of type T.
Core types, constants, and utilities shared across ProvSQL.
The regions of a few requests, in one circuit.
Per-gate metadata stored in the gates MMappedVector.
double prob
Associated probability, NaN when unset.
unsigned info2
General-purpose integer annotation 2.
unsigned long children_idx
Start index of this gate's children in wires.
unsigned info1
General-purpose integer annotation 1.
unsigned long extra_idx
Start index in extra for string data.
unsigned extra_len
Byte length of the string data in extra.
GateInformation(gate_type t, unsigned n, unsigned long i)
Construct a GateInformation with mandatory fields.
unsigned nb_children
Number of children.
gate_type type
Kind of gate (input, plus, times…).
What provsql.check_store() reports about a store.
unsigned long dangling_indices
Mapping entries indexing past the records.
unsigned long nb_gates
Gate records.
unsigned long bad_extra
Records whose extra runs past the extra file.
bool unclean
A file was left marked open-for-writing.
unsigned long nb_mapping
Mapping entries.
unsigned long unreferenced
Records no mapping entry points at.
unsigned long bad_wires
Records whose children run past the wires.
unsigned long next_value
Next index the mapping would assign.
The size of a store, in the three units that matter.
unsigned long extra_bytes
Bytes of variable-length annotation.
unsigned long gates
Gate records.
unsigned long wires
Child wires.