InformationTheory.h File Reference

Information-theoretic readouts over scalar RV sub-circuits: entropy, Kullback-Leibler divergence, mutual information. More...

#include <optional>
#include "GenericCircuit.h"

Include dependency graph for InformationTheory.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Namespaces
namespace	provsql

Functions
double	provsql::computeEntropy (const GenericCircuit &gc, gate_t root, std::optional< gate_t > event)
	Entropy of root in nats: Shannon for a discrete view, differential for a continuous one.
double	provsql::computeKL (const GenericCircuit &gc, gate_t p_root, gate_t q_root)
	Kullback-Leibler divergence KL(P || Q) in nats.
double	provsql::computeMutualInformation (const GenericCircuit &gc, gate_t x_root, gate_t y_root)
	Mutual information I(X; Y) in nats: exactly 0 for structurally independent roots, H(X) / Infinity for identical discrete / continuous roots, and the 2-D histogram plug-in estimate over coupled joint draws otherwise.

Detailed Description

Information-theoretic readouts over scalar RV sub-circuits: entropy, Kullback-Leibler divergence, mutual information.

The exact paths resolve a gate to a closed density view — a bare gate_rv (its family pdf over the integration range), a gate_value / categorical mixture (a finite pmf), or a Bernoulli mixture tree over independent such arms (e.g. the gmm constructor's cascade) — and evaluate the defining integral / sum directly. Entropy of a discrete view is Shannon entropy; of a continuous view, differential entropy (both in nats). Shapes with no density view fall back to Monte Carlo plug-in estimators at the provsql.rv_mc_samples budget (a histogram density for entropy, a 2-D histogram over coupled joint draws for mutual information; KL has no density-free estimator and raises instead).

Definition in file InformationTheory.h.