Query-level correctness of the fused HAVING operator vs the JOIN rewriting #
This file proves the query-level correspondence between the possible-world
semantics of the fused HAVING COUNT(*) operator and its JOIN-based
rewriting, in absorptive commutative m-semirings in which ⊗ distributes
over ⊖.
C = 1(AggQuery.havingSite_count_ge_one): the fusedCOUNT(*) ≥ 1operator agrees – key by key, annotation by annotation – with the duplicate-eliminated key projectionε(Π_{keys}(q)), via the extensional characterizationgroupByKey_eq_dedup_mapof duplicate elimination,Having.havingGroup_coe, andHaving.havingProv_count_ge_one.General
C(Query.joinChain_count_correct): theC-fold self-join chainε(Π_{#0}(joinChain q C))gives every group key the⊕-sumS_{C+1}of the monomials of its(C+1)-element worlds, which is the fusedCOUNT(*) ≥ C + 1predicate provenance. The tie-broken comparison<*of the rewriting is materialized by an occurrence identifier column: the base query has columns (key, value, identifier) and the chain condition compares (value, identifier) pairs lexicographically, so any injective assignment of identifiers within each group – the sole hypothesis – realizes an arbitrary resolution of ties between equal values; duplicate values are fully supported. The statement is per key: a group with fewer thanC + 1occurrences has provenance𝟘on both sides (the fused operator annotates its row with𝟘, the join query has no row for it).
The proof of the general case runs through three layers: the pure chain
algebra (chainAgg, summing to the elementary symmetric sum esymm of
the group's annotations over any strictly increasing enumeration), the
per-key evaluation invariant of the join chain
(joinChain_eval_filter), and the collapse of the fused semantics
(Having.havingProv_count_ge with S_eq_esymm).
=/≤comparisons (Query.joinChainDiff_count_eq_correct,Query.joinChainDiff_count_le_correct): the remaining comparison operators are differences of two≥chains (Q₂^{=C} = Q₂^{≥C} − Q₂^{≥C+1}andQ₂^{≤C} = Q₂^{≥1} − Q₂^{≥C+1}), assembled through theDiffsemantics: since a duplicate-eliminated relation has one row per key, per-key annotation sums commute withDiff(diff_perKeySum), and the group-level content isHaving.G_eq_S_monus_SandHaving.atMost_eq_S_monus_S.
Extensional characterization of duplicate elimination.
groupByKey produces exactly one row per distinct key of the input,
whose annotation is the ⊕-sum of the annotations of the matching
rows.
Dependency graph
Query-level correctness for COUNT(*) ≥ 1. In an absorptive
commutative m-semiring in which ⊗ distributes over ⊖, the fused
HAVING COUNT(*) ≥ 1 site – with its output rows projected to the group
key – computes exactly the duplicate-eliminated key projection
ε(Π_{keys}(q)) of the inner query, which is the C = 1 join-based
query: one row per non-empty group, annotated by the ⊕-sum of the
group's annotations. Stated against any general subquery whose annotated
evaluation is the classical inner query's.
Dependency graph
The chain algebra of the C-fold self-join #
The semantic content of the join chain: the annotated strictly increasing
chains over a multiset of (value, annotation) pairs, and their ⊕-sum,
which is the elementary symmetric sum of the annotations – the multiset
form of the join-side provenance Having.S.
The elementary symmetric sum of a multiset of annotations: the
⊕-sum, over the C-element sub-multisets, of their ⊗-products. This
is the position-free form of the join-side provenance Having.S
(S_eq_esymm).
Equations
- Having.esymm s C = (Multiset.map Multiset.prod (Multiset.powersetCard C s)).sum
Instances For
Dependency graph
Dependency graph
Dependency graph
Having.S over a full position space is the elementary symmetric sum
of the annotation multiset: the join-side provenance only depends on the
multiset of the annotations, not on the position space carrying them.
Dependency graph
The annotated strictly increasing chains of length C + 1 over a
multiset G of (value, annotation) pairs, each chain represented by its
last value and the ⊗-product of its annotations. This is the semantic
content of the C-fold self-join chain of the join-based rewriting.
Equations
- Having.chainAgg G 0 = G
- Having.chainAgg G C.succ = Multiset.map (fun (x : (V × K) × V × K) => (x.2.1, x.1.2 * x.2.2)) (Multiset.filter (fun (x : (V × K) × V × K) => x.1.1 < x.2.1) (Having.chainAgg G C ×ˢ G))
Instances For
Dependency graph
Dependency graph
The last value of a chain is one of the input values.
Dependency graph
Dependency graph
Appending an occurrence whose value dominates every value of G: the
chains over G + {u} are the chains over G together with the chains
ending at u (a u-free chain extended by u, or u alone).
Dependency graph
The chain sum is the elementary symmetric sum. Over a list of
(value, annotation) pairs with strictly increasing values, the ⊕-sum of
the annotations of the strictly increasing chains of length C + 1 is
the elementary symmetric sum of degree C + 1 of the annotations.
Dependency graph
Left coordinate of an appended tuple.
Dependency graph
Right coordinate of an appended tuple, at offset m + i.
Dependency graph
Right coordinate of an appended tuple, at offset m.
Dependency graph
Dependency graph
Dependency graph
Dependency graph
The join chain query and its per-key evaluation #
The C-fold self-join chain of the join-based rewriting, over an arity-3
base query whose columns are (group key, compared value, occurrence
identifier). The identifier column materializes the tie-broken comparison
<*: the chain condition compares (value, identifier) pairs
lexicographically, so any injective assignment of identifiers within each
group realizes an arbitrary resolution of ties between equal values –
duplicate values (and duplicate whole occurrences) are fully supported.
The chain condition relating a new copy of the base query to the last
copy of the chain: equal group keys and lexicographically larger
(value, identifier) pair – the tie-broken comparison <*.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Dependency graph
The C-fold self-join chain: C + 1 copies of the base query, with
consecutive copies related by chainCond. Copy j occupies columns
3j, 3j + 1, 3j + 2.
Instances For
Dependency graph
Dependency graph
The combining map of one chain step: append the new copy's tuple and multiply the annotations.
Equations
- chainCombine C z = (Fin.append z.1.1 z.2.1, z.1.2 * z.2.2)
Instances For
Dependency graph
The (value, identifier) pairs of the group of key a, with their
annotations: the chain-algebra view of one group of the base relation.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Dependency graph
Per-key evaluation of the join chain. Within the group of key a,
the rows of the C-fold chain, viewed through their last (value,
identifier) pair and their annotation, are exactly the annotated strictly
increasing chains of length C + 1 of the group.
Dependency graph
Assembly: the join-based query computes the fused HAVING provenance #
Mapping to the annotation ignores a rebuilt data part.
Dependency graph
Per-key sums over a keyed rebuild of the distinct keys.
Dependency graph
Per-key annotation sums are invariant under duplicate elimination.
Dependency graph
Under global row-distinctness, the group sequence is strictly increasing on its tuple part.
Dependency graph
A strict tuple inequality between arity-3 tuples with equal first column is a strict lexicographic inequality on the (second, third) column pairs.
Dependency graph
The chain-algebra view of a group is the (value, identifier) image of the group sequence.
Dependency graph
Dependency graph
Dependency graph
Query-level correctness of the join-based rewriting, general C.
In an absorptive commutative m-semiring in which ⊗ distributes over
⊖, for every group key g, the ⊕-sum of the annotations that the
join-based query ε(Π_{#0}(joinChain q C)) gives to g equals the
fused HAVING COUNT(*) ≥ C + 1 predicate provenance of the group of g.
The hypothesis hnodup states that the occurrence identifiers of the
base query's third column make its rows pairwise distinct: it is the
formal counterpart of fixing an arbitrary tie-break <* between
occurrences with equal compared values, and is satisfiable for every
instance (annotate each occurrence with a distinct identifier).
The statement is per-key: a group with fewer than C + 1 occurrences
has provenance 𝟘 on both sides – the fused operator gives its row a
𝟘 annotation while the join query simply has no row for it.
Dependency graph
One row per distinct key, at the multiset level: filtering a keyed rebuild of the distinct keys by a key.
Dependency graph
Per-key sums through a key-preserving rebuild of the annotations.
Dependency graph
Per-key sums through Diff. When the left argument of a
difference is duplicate-eliminated (one row per key), the per-key
annotation sum of the difference is the monus of the two per-key sums.
Dependency graph
The join-based query for COUNT(*) ≥ C + 1: project the C-fold
chain to its group key and eliminate duplicates.
Instances For
Dependency graph
Query-level correctness for COUNT(*) = C + 1. The join-based
query Q₂^{≥C+1} − Q₂^{≥C+2} gives every group key the fused
COUNT(*) = C + 1 predicate provenance.
Dependency graph
Query-level correctness for COUNT(*) ≤ C. The join-based query
Q₂^{≥1} − Q₂^{≥C+1} gives every group key the fused COUNT(*) ≤ C
predicate provenance.
Dependency graph
The join-based query realizing COUNT(*) op (C + 1), for each
comparison operator op: chains for ≥ and >, differences of two
chains for ≤, < and =, and the union of the <- and >-queries
for ≠.
Equations
- q.joinCountQuery CompOp.lt x✝ = (joinChainQuery q 0 - joinChainQuery q x✝)
- q.joinCountQuery CompOp.le x✝ = (joinChainQuery q 0 - joinChainQuery q (x✝ + 1))
- q.joinCountQuery CompOp.eq x✝ = (joinChainQuery q x✝ - joinChainQuery q (x✝ + 1))
- q.joinCountQuery CompOp.ne x✝ = ((joinChainQuery q 0 - joinChainQuery q x✝) ⊎ joinChainQuery q (x✝ + 1))
- q.joinCountQuery CompOp.ge x✝ = joinChainQuery q x✝
- q.joinCountQuery CompOp.gt x✝ = joinChainQuery q (x✝ + 1)
Instances For
Dependency graph
Dependency graph
Per-key annotation sums are additive across Query.Sum.
Dependency graph
Query-level correctness of the JOIN rewriting for COUNT(*), for
any comparison operator. For every op ∈ {<, ≤, =, ≠, ≥, >}, every
threshold C + 1 ≥ 1 and every group key, in an absorptive commutative
m-semiring whose ⊗ distributes over ⊖, the join-based query
Query.joinCountQuery q op C gives the group key the fused
COUNT(*) op (C + 1) predicate provenance.