provsql_migrate_mmap.cpp File Reference

Migrate old flat provsql mmap files to the per-database layout. More...

#include <libpq-fe.h>
#include <cassert>
#include <cerrno>
#include <cstdint>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <functional>
#include <iostream>
#include <queue>
#include <stdexcept>
#include <string>
#include <unordered_map>
#include <unordered_set>
#include <utility>
#include <vector>
#include <fcntl.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <unistd.h>

Include dependency graph for provsql_migrate_mmap.cpp:

Go to the source code of this file.

Classes
struct	pg_uuid_t
	UUID structure. More...
struct	UUIDHash
struct	UUIDEq
struct	GateInformation
	Per-gate metadata stored in the gates MMappedVector. More...
struct	OldVecHdr
struct	OldHashSlot
struct	OldTableHdr
class	OldMMapVec
class	OldMMapHash
struct	OldCircuit
struct	NewVecHdr
struct	NewHashSlot
struct	NewTableHdr
struct	Conn
struct	Res
struct	GateData

Typedefs
using	UUIDSet = std::unordered_set<pg_uuid_t, UUIDHash, UUIDEq>

Enumerations
enum	gate_type {   gate_input , gate_plus , gate_times , gate_monus ,   gate_project , gate_zero , gate_one , gate_eq ,   gate_agg , gate_semimod , gate_cmp , gate_delta ,   gate_value , gate_mulinput , gate_update , gate_invalid ,   nb_gate_types }

Functions
static void	writeNewVector (const std::string &path, uint64_t magic, size_t elem_size, const uint8_t *raw, unsigned long nb)
static void	writeNewHashTable (const std::string &path, const std::vector< std::pair< pg_uuid_t, unsigned long > > &entries)
static pg_uuid_t	parseUUID (const char *s)
static std::unordered_map< unsigned long, GateData >	collectReachable (const OldCircuit &old, const UUIDSet &roots)
static void	migrateDatabase (const OldCircuit &old, const std::string &pgdata, Oid db_oid, const UUIDSet &roots)
int	main (int argc, char **argv)

Variables
static constexpr unsigned long	NOTHING = static_cast<unsigned long>(-1)
static constexpr uint64_t	MAGIC_GATES
static constexpr uint64_t	MAGIC_WIRES
static constexpr uint64_t	MAGIC_MAPPING
static constexpr uint64_t	MAGIC_EXTRA

Detailed Description

Migrate old flat provsql mmap files to the per-database layout.

Old format: $PGDATA/provsql_*.mmap (no 16-byte header) New format: $PGDATA/base/<db_oid>/provsql_*.mmap (16-byte header)

For each database that has provsql installed, the tool:

1. Enumerates provenance-tracked tables via libpq.
2. Collects root UUIDs from those tables.
3. BFS-traverses the old flat circuit from those roots.
4. Writes per-database new-format files.

Usage: provsql_migrate_mmap -D <pgdata> -c <connstr>

connstr should point to the postgres / template1 database so the tool can enumerate all databases. The tool then re-connects per database. If connstr already contains "dbname=", append " dbname=\<target\>" to override it (libpq uses the last occurrence).

Gates belonging to tables in more than one database are written to every relevant database. UUID v4 collisions across databases are negligible.

The tool skips any database whose $PGDATA/base/<oid>/provsql_gates.mmap already exists.

Definition in file provsql_migrate_mmap.cpp.

Typedef Documentation

UUIDSet

using UUIDSet = std::unordered_set<pg_uuid_t, UUIDHash, UUIDEq>

Definition at line 68 of file provsql_migrate_mmap.cpp.

Enumeration Type Documentation

gate_type

enum gate_type

Enumerator
gate_input
gate_plus
gate_times
gate_monus
gate_project
gate_zero
gate_one
gate_eq
gate_agg
gate_semimod
gate_cmp
gate_delta
gate_value
gate_mulinput
gate_update
gate_invalid
nb_gate_types

Definition at line 71 of file provsql_migrate_mmap.cpp.

Function Documentation

collectReachable()

std::unordered_map< unsigned long, GateData > collectReachable ( const OldCircuit & old, const UUIDSet & roots )

static

Definition at line 413 of file provsql_migrate_mmap.cpp.

Here is the call graph for this function:

Here is the caller graph for this function:

main()

int main	(	int	argc,
		char **	argv )

Definition at line 559 of file provsql_migrate_mmap.cpp.

Here is the call graph for this function:

migrateDatabase()

void migrateDatabase ( const OldCircuit & old, const std::string & pgdata, Oid db_oid, const UUIDSet & roots )

static

Definition at line 446 of file provsql_migrate_mmap.cpp.

Here is the call graph for this function:

Here is the caller graph for this function:

parseUUID()

pg_uuid_t parseUUID ( const char * s )

static

Definition at line 386 of file provsql_migrate_mmap.cpp.

Here is the caller graph for this function:

writeNewHashTable()

void writeNewHashTable ( const std::string & path, const std::vector< std::pair< pg_uuid_t, unsigned long > > & entries )

static

Definition at line 318 of file provsql_migrate_mmap.cpp.

Here is the caller graph for this function:

writeNewVector()

void writeNewVector ( const std::string & path, uint64_t magic, size_t elem_size, const uint8_t * raw, unsigned long nb )

static

Definition at line 288 of file provsql_migrate_mmap.cpp.

Here is the caller graph for this function:

Variable Documentation

MAGIC_EXTRA

uint64_t MAGIC_EXTRA

staticconstexpr

Initial value:

=
  uint64_t('P')       | uint64_t('v') <<  8 | uint64_t('S') << 16 | uint64_t('E') << 24 |
  uint64_t('x') << 32 | uint64_t('t') << 40 | uint64_t('r') << 48 | uint64_t('a') << 56

Definition at line 259 of file provsql_migrate_mmap.cpp.

MAGIC_GATES

uint64_t MAGIC_GATES

staticconstexpr

Initial value:

=
  uint64_t('P')       | uint64_t('v') <<  8 | uint64_t('S') << 16 | uint64_t('G') << 24 |
  uint64_t('a') << 32 | uint64_t('t') << 40 | uint64_t('e') << 48 | uint64_t('s') << 56

Definition at line 250 of file provsql_migrate_mmap.cpp.

MAGIC_MAPPING

uint64_t MAGIC_MAPPING

staticconstexpr

Initial value:

=
  uint64_t('P')       | uint64_t('v') <<  8 | uint64_t('S') << 16 | uint64_t('M') << 24 |
  uint64_t('a') << 32 | uint64_t('p') << 40 | uint64_t('n') << 48 | uint64_t('g') << 56

Definition at line 256 of file provsql_migrate_mmap.cpp.

MAGIC_WIRES

uint64_t MAGIC_WIRES

staticconstexpr

Initial value:

=
  uint64_t('P')       | uint64_t('v') <<  8 | uint64_t('S') << 16 | uint64_t('W') << 24 |
  uint64_t('i') << 32 | uint64_t('r') << 40 | uint64_t('e') << 48 | uint64_t('s') << 56

Definition at line 253 of file provsql_migrate_mmap.cpp.

NOTHING

unsigned long NOTHING = static_cast<unsigned long>(-1)

staticconstexpr

Definition at line 111 of file provsql_migrate_mmap.cpp.