Documentation

Provenance.MuUnion

μ^∪ from μ^⊎ #

A set recursion μ^∪R.(q₀, q₁) whose recursive term is L(R) ⊎ C, with L reading R once under a projection, a selection, a product, the left side of an apply (and nothing else), is simulated by a bag recursion over pairs of a tuple and a counter: the base is (q₀ ⊎ C) × E^K for a relation E of counters, and a round applies L to the tuple part of every row and moves its counter to every strictly larger one (in the lexicographic order of E^K). A row's counters then form a strictly increasing chain, so the bag recursion ends after |E|^K rounds – it always settles – and its rounds carry, as sets, the rounds of the set recursion, which reach the fixpoint within |F| rounds, F the fixpoint. So ε(Π_tuple(S)) = F as soon as |E|^K ≥ |F| (AggQueryIn.evaluatePlain_simResult).

This module has the construction and its correctness for any counter relation E of non-null values that is large enough. What E can be read off the query is the other half: the values of the instance at the recursion's leaves, its constants and the context, with two constants of the domain (counterRel), and a width K fixed by the query. A count inside the term takes values that no such set enumerates, which is where the order-type lemma enters: the fixpoint is duplicate-free over those values and the counts the recursion computes, of which there are polynomially many (AggQueryIn.exists_card_bound_muSet in Provenance.Genericity), so |F| ≤ |E|^K for a K read off that polynomial.

Linear recursive terms #

The typing of a recursive term admits the recursive relation under a ⊎, with the other side closed; the simulation needs the term without that side – which is the q₁' half of the split the generation pass produces – and reading the relation at a fixed arity.

def AggQueryIn.recLinear {T : Type} (s : String) (n : ℕ) {c m : ℕ} {κ : Fin m → ColKind} {l : List (String × ℕ)} :
AggQueryIn T c m κ l → Prop

A term reading the recursive relation s once, at arity n, and linearly: under a projection, a selection on regular columns, either side of a product or the left side of an apply, with the other side closed. No ⊎: its closed side is the C of q₁ = L ⊎ C.

Equations
Instances For
    Dependency graph

    The rounds of a set recursion reach its fixpoint within its size #

    Counters: strictly increasing chains in a finite order #

    def chainBag {β : Type} (r : β → β → Prop) [DecidableRel r] (Cb : Multiset β) :
    ℕ → Multiset β

    The bag of counters a row can hold after j moves: the whole set at first, then the counters strictly above a counter of the previous round.

    Equations
    Instances For
      Dependency graph

      Replacing the recursive relation by a query #

      def subIdx (s : String) (n : ℕ) (l : List (String × ℕ)) :

      The holes of a term, the recursive relation's taken out.

      Equations
      Instances For
        Dependency graph
        theorem subIdx_append (s : String) (n : ℕ) (l₁ l₂ : List (String × ℕ)) :
        subIdx s n (l₁ ++ l₂) = subIdx s n l₁ ++ subIdx s n l₂
        Dependency graph
        theorem subIdx_self (s : String) (n : ℕ) :
        subIdx s n [(s, n)] = []
        Dependency graph
        theorem subIdx_single_ne {s s' : String} {n n' : ℕ} (h : ¬(s' = s ∧ n' = n)) :
        subIdx s n [(s', n')] = [(s', n')]
        Dependency graph
        def AggQueryIn.castIdx {T : Type} {c m : ℕ} {κ : Fin m → ColKind} {l l' : List (String × ℕ)} (h : l = l') (q : AggQueryIn T c m κ l) :
        AggQueryIn T c m κ l'

        A term, its hole index rewritten.

        Equations
        Instances For
          Dependency graph
          theorem SelOK.sub {T : Type} {s : String} {n : ℕ} {l : List (String × ℕ)} {c m : ℕ} {κ : Fin m → ColKind} {φ : GenPredIn T c κ} (hφ : SelOK l φ) :
          SelOK (subIdx s n l) φ

          Admissibility of a selection's predicate survives taking a hole out.

          Dependency graph
          def AggQueryIn.substRel {T : Type} (s : String) {c n : ℕ} (pv : Fin n → T ⊕ Fin c) {m : ℕ} {κ : Fin m → ColKind} {l : List (String × ℕ)} :
          AggQueryIn T c m κ l → AggQueryIn T c m κ (subIdx s n l)

          The recursive relation replaced by one row read off the context: every hole of its name and arity becomes the parameter leaf reading pv, at the hole's own kinds, so the holes of the result are the term's with that one taken out; nothing else is touched.

          Equations
          Instances For
            Dependency graph

            The counters: a power of a unary relation, read lexicographically #

            def AggQueryIn.pow {T : Type} {c : ℕ} (EQ : AggQueryIn T c 1 (ColKind.allReg 1)) (K : ℕ) :

            E^K, the K-fold product of a unary relation.

            Equations
            Instances For
              Dependency graph

              The values of a unary relation.

              Equations
              Instances For
                Dependency graph

                The lexicographic order on the counters is a total order #

                def ascLe {T : Type} [ValueType T] (p : ℕ) (x y : Tuple T p) :

                The relation the counters are sorted by.

                Equations
                Instances For
                  Dependency graph
                  def ascLt {T : Type} [ValueType T] (p : ℕ) (x y : Tuple T p) :

                  And the strict one the rounds move along.

                  Equations
                  Instances For
                    Dependency graph
                    @[instance_reducible]
                    Equations
                    Dependency graph
                    @[instance_reducible]
                    Equations
                    Dependency graph
                    instance instIsTransTupleAscLe {T : Type} [ValueType T] (p : ℕ) :
                    IsTrans (Tuple T p) (ascLe p)
                    Dependency graph
                    Dependency graph
                    Dependency graph
                    Dependency graph

                    The construction #

                    def AggQueryIn.dropLeft {T : Type} (n K : ℕ) :
                    Tuple T (n + K + (n + K)) → Tuple T (n + K)

                    The columns of the right side of the apply the round is built on.

                    Equations
                    Instances For
                      Dependency graph
                      def AggQueryIn.leftPart {T : Type} (n K : ℕ) (u : Tuple T (n + K)) :
                      Tuple T n

                      The tuple part of a row of the simulation.

                      Equations
                      Instances For
                        Dependency graph
                        def AggQueryIn.rightPart {T : Type} (n K : ℕ) (u : Tuple T (n + K)) :
                        Tuple T K

                        And its counter.

                        Equations
                        Instances For
                          Dependency graph
                          def AggQueryIn.rowPv {T : Type} {c : ℕ} (n K : ℕ) :
                          Fin n → T ⊕ Fin (n + K + c)

                          The current row's tuple part, read off the outer columns of the apply: the parameters a hole of the recursive term is replaced by.

                          Equations
                          Instances For
                            Dependency graph
                            def AggQueryIn.ltSel {T : Type} [ValueType T] {c : ℕ} (n K : ℕ) :
                            GenPredIn T (n + K + c) (ColKind.allReg K)

                            The counter moves up: the row's counter is strictly above the current row's, lexicographically.

                            Equations
                            Instances For
                              Dependency graph
                              def AggQueryIn.simBody {T : Type} [ValueType T] {c : ℕ} (s s' : String) (n K : ℕ) (EQ : AggQueryIn T c 1 (ColKind.allReg 1)) (L : AggQueryIn T c n (ColKind.allReg n) [(s, n)]) :
                              AggQueryIn T c (n + K) (ColKind.allReg (n + K)) [(s', n + K)]

                              One round of the simulation: for each row of the recursive relation, the term at the row's tuple part, paired with every counter above the row's.

                              Equations
                              • One or more equations did not get rendered due to their size.
                              Instances For
                                Dependency graph

                                The simulation, and what it computes #

                                def AggQueryIn.pairRel {T : Type} {n K : ℕ} (A : Multiset (Tuple T n)) (M : Multiset (Tuple T K)) :
                                Multiset (Tuple T (n + K))

                                The product of two relations, as the bag of appended rows.

                                Equations
                                Instances For
                                  Dependency graph
                                  def AggQueryIn.sim {T : Type} [ValueType T] {c : ℕ} (s s' : String) (n K : ℕ) (EQ : AggQueryIn T c 1 (ColKind.allReg 1)) (BQ : AggQueryIn T c n (ColKind.allReg n)) (L : AggQueryIn T c n (ColKind.allReg n) [(s, n)]) :
                                  AggQueryIn T c (n + K) (ColKind.allReg (n + K))

                                  The simulation: a bag recursion over rows pairing a tuple with a counter, from (q₀ ⊎ C) × E^K, each round applying the term to the tuple part and moving the counter up.

                                  Equations
                                  Instances For
                                    Dependency graph
                                    theorem AggQueryIn.dedupG_kinds (n : ℕ) :
                                    (fun (j : Fin (n + 0)) => Fin.append (ColKind.allReg n) (fun (x : Fin 0) => ColKind.agg) j) = ColKind.allReg (n + 0)
                                    Dependency graph
                                    def AggQueryIn.dedupG {T : Type} {c n : ℕ} (q : AggQueryIn T c n (ColKind.allReg n)) :

                                    Duplicate elimination as a grouping by every column with no aggregate: the shape the bases without ε use.

                                    Equations
                                    Instances For
                                      Dependency graph
                                      def AggQueryIn.simResult {T : Type} [ValueType T] {c : ℕ} (s s' : String) (n K : ℕ) (EQ : AggQueryIn T c 1 (ColKind.allReg 1)) (BQ : AggQueryIn T c n (ColKind.allReg n)) (L : AggQueryIn T c n (ColKind.allReg n) [(s, n)]) :

                                      Its tuple parts, without duplicates: the set recursion's value.

                                      Equations
                                      • One or more equations did not get rendered due to their size.
                                      Instances For
                                        Dependency graph
                                        noncomputable def AggQueryIn.rowTerm {T : Type} [ValueType T] {c : ℕ} (s : String) (n : ℕ) (L : AggQueryIn T c n (ColKind.allReg n) [(s, n)]) (D : Database T) (γ : Fin c → T) :
                                        Tuple T n → Multiset (Tuple T n)

                                        The term applied to one row of the recursive relation.

                                        Equations
                                        Instances For
                                          Dependency graph
                                          noncomputable def AggQueryIn.simStep {T : Type} [ValueType T] {c : ℕ} (s s' : String) (n K : ℕ) (EQ : AggQueryIn T c 1 (ColKind.allReg 1)) (L : AggQueryIn T c n (ColKind.allReg n) [(s, n)]) (D : Database T) (γ : Fin c → T) :
                                          Multiset (Tuple T (n + K)) → Multiset (Tuple T (n + K))

                                          One round of the simulation, as a function of the recursive relation.

                                          Equations
                                          Instances For
                                            Dependency graph
                                            noncomputable def AggQueryIn.counters {T : Type} [ValueType T] {c : ℕ} (EQ : AggQueryIn T c 1 (ColKind.allReg 1)) (K : ℕ) (D : Database T) (γ : Fin c → T) :

                                            The counters.

                                            Equations
                                            Instances For
                                              Dependency graph

                                              The set recursion's rounds, as the term iterated on the base #

                                              def bagStep {α : Type} (B : Multiset α) (L₁ : α → Multiset α) (X : Multiset α) :

                                              The step of a set recursion μ^∪R.(B, L(R)) with L linear, as a bag before the duplicate elimination.

                                              Equations
                                              Instances For
                                                Dependency graph

                                                The simulation computes the set recursion #

                                                theorem AggQueryIn.evaluatePlain_simResult {T : Type} [ValueType T] {c : ℕ} (s s' : String) (n K : ℕ) (EQ : AggQueryIn T c 1 (ColKind.allReg 1)) (BQ qC q₀ : AggQueryIn T c n (ColKind.allReg n)) (L q₁ : AggQueryIn T c n (ColKind.allReg n) [(s, n)]) (hL : recLinear s n L) (hLs : noMention s' L) (hEs : noMention s' EQ) (hC : noMention s qC) (D : Database T) (γ : Fin c → T) (hset : (MuSet s q₀ q₁).SettlesAt D γ) (hsplit : ∀ (X : Relation T n), q₁.SettlesAt (Database.assign s X D) γ → q₁.evaluatePlain (Database.assign s X D) γ = L.evaluatePlain (Database.assign s X D) γ + qC.evaluatePlain (Database.assign s X D) γ) (hB : BQ.evaluatePlain D γ = q₀.evaluatePlain D γ + qC.evaluatePlain D γ) (hE : ∀ e ∈ unaryVals (EQ.evaluatePlain D γ), ValueType.isNull e = false) (hcard : Multiset.card ((MuSet s q₀ q₁).evaluatePlain D γ) ≤ (unaryVals (EQ.evaluatePlain D γ)).toFinset.card ^ K) :
                                                (simResult s s' n K EQ BQ L).evaluatePlain D γ = (MuSet s q₀ q₁).evaluatePlain D γ ∧ ∀ (j : ℕ) (t : Tuple T n), j < (EQ.counters K D γ).toFinset.card → t ∈ (fun (X : Multiset (Tuple T n)) => X.bind (rowTerm s n L D γ))^[j] (BQ.evaluatePlain D γ) → Multiset.Mem ((MuSet s q₀ q₁).evaluatePlain D γ) t

                                                μ^∪ from μ^⊎ (the correctness of the construction): for a set recursion μ^∪R.(q₀, q₁) whose term splits as L ⊎ C with L linear, a base query computing q₀ ⊎ C, and a counter relation E of non-null values with |E|^K at least the size of the fixpoint, the simulation's tuples are the fixpoint, wherever the recursion is defined.

                                                Dependency graph

                                                The counter relation #

                                                The simulation runs its counter over E^K for a unary relation E of non-null values with |E|^K at least the size of the fixpoint. E is computed from what the fixpoint's values can be – the values of the base (so of q₀ and of C), of the relations the term reads off the instance, the term's constants and the context's values – with two constants of the domain so that it holds at least two values, and a selection keeping the non-null ones.

                                                The sum of a list of queries.

                                                Equations
                                                Instances For
                                                  Dependency graph
                                                  def AggQueryIn.colRel {T : Type} {c n : ℕ} (q : AggQueryIn T c n (ColKind.allReg n)) (i : Fin n) :

                                                  Column i of a query, as a unary relation.

                                                  Equations
                                                  Instances For
                                                    Dependency graph
                                                    def AggQueryIn.constRel {T : Type} {c : ℕ} (a : T) :

                                                    A constant, as a one-row unary relation.

                                                    Equations
                                                    Instances For
                                                      Dependency graph
                                                      def AggQueryIn.outerRel {T : Type} {c : ℕ} (j : Fin c) :

                                                      A context value, as a one-row unary relation.

                                                      Equations
                                                      Instances For
                                                        Dependency graph

                                                        x IS NOT NULL, spelled x = x: the comparison is unknown exactly at the null.

                                                        Equations
                                                        Instances For
                                                          Dependency graph
                                                          def AggQueryIn.counterParts {T : Type} {c n : ℕ} (B : AggQueryIn T c n (ColKind.allReg n)) (ls : List (ℕ × String)) (cs : List T) :

                                                          The unary relations the counter relation is the union of.

                                                          Equations
                                                          • One or more equations did not get rendered due to their size.
                                                          Instances For
                                                            Dependency graph
                                                            def AggQueryIn.counterRel {T : Type} [ValueType T] {c n : ℕ} (B : AggQueryIn T c n (ColKind.allReg n)) (ls : List (ℕ × String)) (cs : List T) :

                                                            The counter relation: the non-null values among the columns of B, the columns of the relations ls names, the constants cs and the context's values.

                                                            Equations
                                                            Instances For
                                                              Dependency graph

                                                              What the simulation is made of #

                                                              The structural facts the pass needs of the simulation: which classes it uses, that its recursion is restricted, its signature, its schema, its names. Each is read off its parts.

                                                              Bridges for the pass #

                                                              The set recursion replaced #

                                                              The simulation as the pass needs it: where the target classes hold no μ^∪, a set recursion is replaced by the simulation – its base the rewritten base and closed part summed, its term the rewritten body, its counter the counter relation over the values the fixpoint can hold – and the unions the counter relation is built with go by the bag recursion afterwards.

                                                              The whole algebra with one recursion #

                                                              The basis of the whole algebra in the minimal universe: the construction's classes and the windows, {∅, Π, σ, apply, γ, win, μ^⊎}. basisRecOps_generatesOnRec says it generates, and Provenance.RecursionBasisTheorem that no proper subset does.

                                                              Equations
                                                              Instances For
                                                                Dependency graph
                                                                theorem basisRecOps_generatesOnRec {T : Type} [ValueType T] (sch : Schema) (cnt : SeqAggFunc T) (hcnt : cnt.IsCount) :

                                                                μ^∪ is not needed, for every restricted term: over a schema declaring finitely many names, every query of the whole algebra in the minimal universe with restricted recursions is matched over ∅, Π, σ, the apply, γ, win and μ^⊎ – defined wherever the query is and equal to it there. A set recursion is simulated by a bag recursion whose counter runs over the instance's values at the recursion's leaves, the recursion's constants and its context: the fixpoint is duplicate-free over those values and the counts the recursion computes, of which there are polynomially many (AggQueryIn.exists_card_bound_muSet, off the order types of Provenance.Genericity), so a counter of a width fixed by the query covers it. With the necessary classes (required_mem_of_generatesOnRec), a basis of the whole algebra in the minimal universe is ∅, Π, γ, μ^⊎, one of σ and −, one of × and the apply, and win (Provenance.WinRow); μ^∪ is in none.

                                                                Dependency graph

                                                                And without the window class, for the windows an apply derives (evaluatePlain_winByApplyOf): the six classes ∅, Π, σ, the apply, γ and μ^⊎ match every query of the whole algebra in the minimal universe whose windows are derivable (AggQueryIn.WinsDerivable).

                                                                Dependency graph