Why Tensors? - UTEP CS
Objective of Science . . . Partial Orders . . . Numerical . . .
Why Tensors? Olga Kosheleva, Martine Ceberio, and Vladik Kreinovich
Need to Combine . . . Mathematical . . . Towards a General . . . Towards an . . . What If . . .
University of Texas at El Paso El Paso, Texas 79968, USA [email protected], [email protected] [email protected]
Acknowledgments Title Page
JJ
II
J
I
Page 1 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
1.
Objective of Science and Engineering • One of the main objectives: help people select decisions which are the most beneficial to them.
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . .
• To make these decisions,
Towards a General . . .
– we must know people’s preferences,
Towards an . . .
– we must have the information about different events – possible consequences of different decisions, and
What If . . . Acknowledgments Title Page
– we must also have information about the degree of certainty ∗ (since information is never absolutely accurate and precise).
JJ
II
J
I
Page 2 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
2.
Partial Orders Naturally Appear in Many Application Areas • Reminder: we need info re preferences, events, and degrees of certainty.
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . .
• All these types of information naturally lead to partial orders: – For preferences, a < b means that b is preferable to a. ∗ This relation is used in decision theory. – For events, a < b means that a can influence b. ∗ This causality relation is used in space-time physics. – For degrees of certainty, a < b means that a is less certain than b.
Towards an . . . What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 3 of 25
Go Back
Full Screen
∗ This relation is used in logics describing uncertainty – such as fuzzy logic.
Close
Quit
Objective of Science . . .
3.
Numerical Characteristics Related to Partial Orders
+ An order is a natural way of describing a relation.
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . .
− Orders are difficult to process, since most data processing algorithms process numbers. • Natural idea: use numerical characteristics to describe the orders.
Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
• Fact: this idea is used in all three application areas: – in decision making, utility describes preferences:
JJ
II
J
I
a < b if and only if u(a) < u(b); Page 4 of 25
– in space-time physics, metric (and time coordinates) describes causality relation; – in logic and soft constraints, numbers from the interval [0, 1] are used to describe degrees of certainty.
Go Back
Full Screen
Close
Quit
Objective of Science . . .
4.
Need to Combine Numerical Characteristics: Emergence of Polynomial Aggregation Formulas • In decision making, we need to combine utilities u1 , . . . , un of different participants.
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . .
– Nobelist Josh Nash showed that reasonable conditions lead to u = u1 · . . . · un . • In space-time geometry, we need to combine coordinates xi into a metric. – Reasonable conditions lead to polynomial metrics s2 = c2 ·(x0 −x00 )2 −(x1 −x01 )2 −(x2 −x02 )2 −(x3 −x03 )2 ; s4 = (x1 − x01 ) · (x2 − x02 ) · (x3 − x03 ) · (x4 − x04 ). • In fuzzy logic, we must combine degrees of certainty di in Ai into a degree d for A1 & A2 . – Reasonable conditions lead to polynomial functions like d = d1 · d2 .
Towards an . . . What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 5 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
5.
Mathematical Observation: Polynomial Formulas Are Tensor-Related • Fact: in many areas, we have a general polynomial dependence f (x1 , . . . , xn ) = f0 + n X fi · xi + i=1 n n XX
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
fij · xi · xj +
JJ
II
J
I
i=1 j=1 n X n X n X i=1 j=1 k=1
fijk · xi · xj · xk + ...
• In mathematical terms: to describe this dependence, we need a finite set of tensors f0 , fi , fij , fijk , . . .
Page 6 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
6.
Towards a General Justification of Polynomial (Tensor) Formulas • Fact: similar polynomials appear in different application areas.
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . .
• Reasonable conclusion: there must be a common reason behind them. • What we do: we provide such a general reason.
Towards an . . . What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 7 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
7.
Class of Functions • Objective: find a finite-parametric class F of analytical functions f (x1 , . . . , xn ).
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . .
• Meaning: f (x1 , . . . , xn ) approximate the actual complex aggregation function. • Reasonable requirement: this class F is invariant with respect to addition and multiplication by a constant.
Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
• Conclusion: the class F is a (finite-dimensional) linear space of functions. • Meaning: invariance w.r.t. multiplication by a constant corresponds to the choice of a measuring unit. • If we replace the original measuring unit by a one which is λ times smaller, then all the numerical values ·λ: f (x1 , . . . , xn ) is replaced with λ · f (x1 , . . . , xn ).
JJ
II
J
I
Page 8 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
8.
Similar Scale-Invariance for the Inputs xi • Similarly: in all three areas, the numerical values xi are defined modulo the choice of a measuring unit.
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . .
– If we replace the original measuring unit by a one which is λ times smaller, – then all the numerical values get multiplied by this factor λ: xi is replaced with λ · xi . • Conclusion: it is reasonable to require that the finitedimensional linear space F be invariant with respect to such re-scalings:
Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 9 of 25
– if f (x1 , . . . , xn ) ∈ F , – then for every λ > 0, the function def
Go Back
Full Screen
fλ (x1 , . . . , xn ) = f (λ · x1 , . . . , λ · xn ) Close
also belongs to the family F . Quit
Objective of Science . . .
9.
Definition and the Main Result
Definition. Let n be an arbitrary integer. We say that a finite-dimensional linear space F of analytical functions of n variables is scale-invariant if for every f ∈ F and for every λ > 0, the function
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . . Towards an . . .
def
fλ (x1 , . . . , xn ) = f (λ · x1 , . . . , λ · xn ) also belongs to the family F . Main result. For every scale-invariant finite-dimensional linear space F of analytical functions, every element f ∈ F is a polynomial.
What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 10 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
10.
Proof (Part 1)
Partial Orders . . .
• Let F be a scale-invariant finite-dimensional linear space F of analytical functions.
Numerical . . . Need to Combine . . . Mathematical . . .
• Let f (x1 , . . . , xn ) be a function from this family F . • By definition, an analytical function f (x1 , . . . , xn ) is an infinite series consisting of monomials m(x1 , . . . , xn ): m(x1 , . . . , xn ) = ai1 ...in ·
xi11
· ... ·
xinn .
• For each such term, by its total order, we will understand the sum i1 + . . . + in . – if we multiply each input of this monomial by λ, – then the value of the monomial is multiplied by λk :
Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 11 of 25
Go Back
m(λ · x1 , . . . λ · xn ) = ai1 ...in · (λ · x1 )i1 · . . . · (λ · xn )in =
Full Screen
λi1 +...+in · ai1 ...in · xi11 · . . . · xinn = λk · m(x1 , . . . , xn ).
Close
Quit
Objective of Science . . .
11.
Proof (Part 2)
Partial Orders . . .
• Reminder: f (x1 , . . . , xn ) is a sum of monomials m(x1 , . . . , xn ) = ai1 ...in ·
xi11
· ... ·
xinn .
Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . .
• For each monomial, by its order, we will understand the sum k = i1 + . . . + in . • For each order k, there are finitely many possible combinations of integers i1 , . . . , in for which i1 +. . .+in = k. • So, there are finitely many possible monomials of the order k. • Let Pk (x1 , . . . , xn ) denote the sum of all the monomials of order k in the expansion of f (x1 , . . . , xn ). • Then, we have f (x1 , . . . , xn ) = P0 +P1 (x1 , . . . , xn )+P2 (x1 , x2 , . . . , xn )+. . .
Towards an . . . What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 12 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
12.
Proof (Part 3)
• f (x) = P0 +P1 (x1 , . . . , xn )+P2 (x1 , . . . , xn )+. . . , where Pk (x1 , . . . , xn ) is the sum of monomials of order k. • Some of the sums Pk may be zeros – if the expansion of f has no monomials of the corresponding order. • Let k0 be the first index for which the term Pk0 (x1 , . . . , xn ) is not identically 0. Then,
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
f (x1 , . . . , xn ) = Pk0 (x1 , . . . , xn )+Pk0 +1 (x1 , . . . , xn )+. . . • Since the family F is scale-invariant, it also contains fλ (x1 , . . . , xn ) = f (λ · x1 , . . . , λ · xn ). • At this re-scaling, each term Pk is multiplied by λk . • Thus, we get fλ (x) = λk0 ·Pk0 (x1 , . . . , xn )+λk0 +1 ·Pk0 +1 (x1 , . . . , xn )+. . .
JJ
II
J
I
Page 13 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
13.
Proof (Part 4)
• Proven: fλ (x) = λk0 · Pk0 (x) + λk0 +1 · Pk0 +1 (x) + . . . ∈ F. • Since F is a linear space, it also contains a function λ−k0 · fλ (x) = Pk0 (x) + λ · Pk0 +1 (x) + . . .
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . . Towards an . . .
• Since F is finite-dimensional, it is closed under turning to a limit.
What If . . . Acknowledgments
• In the limit λ → 0, we conclude that the term Pk0 (x) also belongs to the family F : Pk0 (x) ∈ F .
JJ
II
• Since F is a linear space, this means that the difference
J
I
f (x) − Pk0 (x) = Pk0 +1 (x) + Pk0 +2 (x) + . . . ∈ F. • Let k1 be the first index k1 > k0 for which the term Pk1 (x) is not identically 0. • Then we can similarly conclude that the term Pk1 (x) also belongs to the family F , etc.
Title Page
Page 14 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
14.
Proof (Conclusion)
• We can therefore conclude that: – for every index k for which Pk (x) 6≡ 0, – this term Pk (x) also belongs to the family F . • Fact: monomials of different total order are linearly independent: – if there were infinitely many non-zero terms Pk in the expansion of the function f (x), – we would have infinitely many linearly independent function in the family F – which contradicts to our assumption that the family F is a finite-dimensional linear space. • So, there are only finitely many non-zero Pk . • Hence, f (x) is a sum of finitely many monomials – i.e., a polynomial.
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 15 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
15.
Towards an Alternative Justification Based on Optimality
• Idea: we would like to select the optimal finite-dimensional family of analytical functions F .
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . .
• What is an optimality criterion: when we can decide – whether F is better than F 0 (denoted F 0 ≺ F ) – or F 0 is better than F (F ≺ F 0 )
What If . . . Acknowledgments
– or F 0 is of the same quality as F (denoted F ≡ F 0 ). 0
Towards an . . .
0
• E.g.: numerical criterion F ≺ F ⇔ J(F ) < J(F ). • More general case: – when J(F ) = J(F 0 ), e.g., for average approximation accuracy J(F ), – we can still choose between F and F 0 based on some other criteria J 0 (e.g., computational simplicity).
Title Page
JJ
II
J
I
Page 16 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
16.
Towards General Description of Optimality
Partial Orders . . . Numerical . . .
• Reminder:
Need to Combine . . .
0
– when J(F ) = J(F ), e.g., for average approximation accuracy J(F ),
Mathematical . . . Towards a General . . .
– we can still choose between F and F 0 based on some other criteria J 0 (e.g., computational simplicity).
Towards an . . . What If . . . Acknowledgments
• The resulting criterion is non-numerical:
Title Page
0
0
0
0
0
0
F ≺ F ⇔ J(F ) < J(F )∨(J(F ) = J(F ) & J (F ) < J (F ). • General definition: a (pre)-ordering relation . • Natural requirement: which operation is better should be not depend on the choice of measuring unit: F ≺ F 0 ⇔ Fλ ≺ Fλ0 , where Fλ = {fλ : f ∈ F }.
JJ
II
J
I
Page 17 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
17.
Optimization Approach: Definitions
• We consider the set A of all finite-dimensional spaces of analytical functions.
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . .
• By an optimality criterion, we mean a pre-ordering (i.e., a transitive, reflexive relation) on the set A. • An optimality criterion on the class of all finitedimensional is called scale-invariant if – for all F , F 0 , and λ 6= 0, – F F 0 implies Fλ Fλ0 . • An optimality criterion is called final if there exists
Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 18 of 25
– one and only one space F Go Back
– that is preferable to all the others, i.e., for which F 0 F for all F 0 6= F .
Full Screen
Close
Quit
Objective of Science . . .
18.
Why Final Criterion: Motivations
• Reminder: an optimality criterion is final if there exists one and only one optimal space F .
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . .
• If no space is optimal relative to some criterion, then this criterion is useless. • If the criterion selects several spaces F as equally good, then we can also optimize something else.
Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
• Example: 0
– if F and F have the same average approximation accuracy, – we can select, among them, the one which is easier to compute. • Thus, such criteria can be adjusted. • So, for the final criterion, the optimal space is unique.
JJ
II
J
I
Page 19 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
19.
Optimization Approach: Main Result
• Condition: Fopt is optimal w.r.t. some scale-invariant and final optimality criterion.
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . .
• Conclusion: all elements of Fopt are polynomials.
Towards a General . . .
• Proof:
Towards an . . .
– optimality means F Fopt for all F ∈ A; – in particular, Fλ−1 Fopt for all F ∈ A; – due to scale-invariance of , we have F (Fopt )λ for all F ∈ A; – thus, (Fopt )λ is optimal; – since there is only one optimal space, we have (Fopt )λ = Fopt ; – thus, the space Fopt is scale-invariant; – we already know that in this case, all f ∈ Fopt are polynomials.
What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 20 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
20.
What If f (x1 , . . . , xn ) Is Only Smooth?
Partial Orders . . . Numerical . . .
Definition. Let n be an arbitrary integer. We say that a finite-dimensional linear space F of smooth functions of n variables is affine-invariant if for every f ∈ F and for every linear transormation T : Rn → Rn , the function
Need to Combine . . . Mathematical . . . Towards a General . . . Towards an . . .
def
fT (x) = f (T x) also belongs to the family F . Main result. For every affine-invariant finite-dimensional linear space F of smooth functions, every element f ∈ F is a polynomial.
What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 21 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
21.
Proof: Main Ideas
Partial Orders . . .
• Let f1 (x), . . . , fm (x) be the basis of F . • For every i ≤ m, for every variable xj and for every λ > 0, we have fi (x1 , . . . , xj−1 , λ · xj , xj+1 , . . . , xn ) ∈ F.
Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . . Towards an . . . What If . . .
• Since fi form a basis, for some cik (λ), we have
Acknowledgments Title Page
fi (x1 , . . . , xj−1 , λ · xj , xj+1 , . . . , xn ) = m X
cik (λ) · fk (x1 , . . . , xj−1 , xj , xj+1 , . . . , xn ).
k=1
JJ
II
J
I
Page 22 of 25
• Differentiating both sides by λ, we get
Go Back
m
X ∂fi xj · = cjk · fk . ∂xj k=1
Full Screen
Close
Quit
Objective of Science . . .
22.
Proof (cont-d)
Partial Orders . . . Numerical . . .
m
X ∂fi • Reminder: xj · = cjk · fk . ∂xj
Need to Combine . . .
k=1
Mathematical . . .
m
X ∂fi = cik · fk . • For Xj = ln(xj ), we have ∂Xj def
k=1
• In terms of Xj , we have a system of linear ODEs with constant coefficients. • A general solution to such a system is a linear combination of terms xαj
• exp(α · Xj ) = (with possible complex α) and p • Xj · exp(α · Xj ) = xαj · lnp (xj ). • A general linear transformation leads to different terms – except when we have xαj for integer α ≥ 0. • Thus, every f ∈ F is a polynomial in each variable – hence a polynomial.
Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 23 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
23.
Acknowledgments
This work was supported in part
Partial Orders . . . Numerical . . . Need to Combine . . .
• by the National Science Foundation grants HRD-0734825 and DUE-0926721, • by Grant 1 T36 GM078000-01 from the National Institutes of Health, ˇ • by Grant MSM 6198898701 from MSMT of Czech Republic, • by Grant 5015 “Application of fuzzy logic with operators in the knowledge based systems” from the Science and Technology Centre in Ukraine (STCU), funded by European Union, and • by the International Finsler Foundation.
Mathematical . . . Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 24 of 25
Go Back
Full Screen
Close
Quit
24.
References Related to Other Application Areas
Objective of Science . . . Partial Orders . . .
• Luce, R. D., Raiffa, R.: Games and decisions: introduction and critical survey, Dover, New York, 1989. • Nguyen H. T., Kosheleva O., Kreinovich, V.: “Decision Making Beyond Arrow’s Impossibility Theorem”, International Journal of Intelligent Systems, 24(1), 27– 47 (2009).
Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . . Towards an . . . What If . . . Acknowledgments
Title Page
• Nguyen, H. T., Walker, E. A.: A First Course in Fuzzy Logic, Chapman & Hall/CRC Press, Boca Raton, Florida, 2006.
JJ
II
J
I
Page 25 of 25 Go Back Full Screen Close Quit
Why Tensors? Olga Kosheleva, Martine Ceberio, and Vladik Kreinovich
Need to Combine . . . Mathematical . . . Towards a General . . . Towards an . . . What If . . .
University of Texas at El Paso El Paso, Texas 79968, USA [email protected], [email protected] [email protected]
Acknowledgments Title Page
JJ
II
J
I
Page 1 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
1.
Objective of Science and Engineering • One of the main objectives: help people select decisions which are the most beneficial to them.
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . .
• To make these decisions,
Towards a General . . .
– we must know people’s preferences,
Towards an . . .
– we must have the information about different events – possible consequences of different decisions, and
What If . . . Acknowledgments Title Page
– we must also have information about the degree of certainty ∗ (since information is never absolutely accurate and precise).
JJ
II
J
I
Page 2 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
2.
Partial Orders Naturally Appear in Many Application Areas • Reminder: we need info re preferences, events, and degrees of certainty.
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . .
• All these types of information naturally lead to partial orders: – For preferences, a < b means that b is preferable to a. ∗ This relation is used in decision theory. – For events, a < b means that a can influence b. ∗ This causality relation is used in space-time physics. – For degrees of certainty, a < b means that a is less certain than b.
Towards an . . . What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 3 of 25
Go Back
Full Screen
∗ This relation is used in logics describing uncertainty – such as fuzzy logic.
Close
Quit
Objective of Science . . .
3.
Numerical Characteristics Related to Partial Orders
+ An order is a natural way of describing a relation.
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . .
− Orders are difficult to process, since most data processing algorithms process numbers. • Natural idea: use numerical characteristics to describe the orders.
Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
• Fact: this idea is used in all three application areas: – in decision making, utility describes preferences:
JJ
II
J
I
a < b if and only if u(a) < u(b); Page 4 of 25
– in space-time physics, metric (and time coordinates) describes causality relation; – in logic and soft constraints, numbers from the interval [0, 1] are used to describe degrees of certainty.
Go Back
Full Screen
Close
Quit
Objective of Science . . .
4.
Need to Combine Numerical Characteristics: Emergence of Polynomial Aggregation Formulas • In decision making, we need to combine utilities u1 , . . . , un of different participants.
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . .
– Nobelist Josh Nash showed that reasonable conditions lead to u = u1 · . . . · un . • In space-time geometry, we need to combine coordinates xi into a metric. – Reasonable conditions lead to polynomial metrics s2 = c2 ·(x0 −x00 )2 −(x1 −x01 )2 −(x2 −x02 )2 −(x3 −x03 )2 ; s4 = (x1 − x01 ) · (x2 − x02 ) · (x3 − x03 ) · (x4 − x04 ). • In fuzzy logic, we must combine degrees of certainty di in Ai into a degree d for A1 & A2 . – Reasonable conditions lead to polynomial functions like d = d1 · d2 .
Towards an . . . What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 5 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
5.
Mathematical Observation: Polynomial Formulas Are Tensor-Related • Fact: in many areas, we have a general polynomial dependence f (x1 , . . . , xn ) = f0 + n X fi · xi + i=1 n n XX
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
fij · xi · xj +
JJ
II
J
I
i=1 j=1 n X n X n X i=1 j=1 k=1
fijk · xi · xj · xk + ...
• In mathematical terms: to describe this dependence, we need a finite set of tensors f0 , fi , fij , fijk , . . .
Page 6 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
6.
Towards a General Justification of Polynomial (Tensor) Formulas • Fact: similar polynomials appear in different application areas.
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . .
• Reasonable conclusion: there must be a common reason behind them. • What we do: we provide such a general reason.
Towards an . . . What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 7 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
7.
Class of Functions • Objective: find a finite-parametric class F of analytical functions f (x1 , . . . , xn ).
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . .
• Meaning: f (x1 , . . . , xn ) approximate the actual complex aggregation function. • Reasonable requirement: this class F is invariant with respect to addition and multiplication by a constant.
Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
• Conclusion: the class F is a (finite-dimensional) linear space of functions. • Meaning: invariance w.r.t. multiplication by a constant corresponds to the choice of a measuring unit. • If we replace the original measuring unit by a one which is λ times smaller, then all the numerical values ·λ: f (x1 , . . . , xn ) is replaced with λ · f (x1 , . . . , xn ).
JJ
II
J
I
Page 8 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
8.
Similar Scale-Invariance for the Inputs xi • Similarly: in all three areas, the numerical values xi are defined modulo the choice of a measuring unit.
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . .
– If we replace the original measuring unit by a one which is λ times smaller, – then all the numerical values get multiplied by this factor λ: xi is replaced with λ · xi . • Conclusion: it is reasonable to require that the finitedimensional linear space F be invariant with respect to such re-scalings:
Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 9 of 25
– if f (x1 , . . . , xn ) ∈ F , – then for every λ > 0, the function def
Go Back
Full Screen
fλ (x1 , . . . , xn ) = f (λ · x1 , . . . , λ · xn ) Close
also belongs to the family F . Quit
Objective of Science . . .
9.
Definition and the Main Result
Definition. Let n be an arbitrary integer. We say that a finite-dimensional linear space F of analytical functions of n variables is scale-invariant if for every f ∈ F and for every λ > 0, the function
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . . Towards an . . .
def
fλ (x1 , . . . , xn ) = f (λ · x1 , . . . , λ · xn ) also belongs to the family F . Main result. For every scale-invariant finite-dimensional linear space F of analytical functions, every element f ∈ F is a polynomial.
What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 10 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
10.
Proof (Part 1)
Partial Orders . . .
• Let F be a scale-invariant finite-dimensional linear space F of analytical functions.
Numerical . . . Need to Combine . . . Mathematical . . .
• Let f (x1 , . . . , xn ) be a function from this family F . • By definition, an analytical function f (x1 , . . . , xn ) is an infinite series consisting of monomials m(x1 , . . . , xn ): m(x1 , . . . , xn ) = ai1 ...in ·
xi11
· ... ·
xinn .
• For each such term, by its total order, we will understand the sum i1 + . . . + in . – if we multiply each input of this monomial by λ, – then the value of the monomial is multiplied by λk :
Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 11 of 25
Go Back
m(λ · x1 , . . . λ · xn ) = ai1 ...in · (λ · x1 )i1 · . . . · (λ · xn )in =
Full Screen
λi1 +...+in · ai1 ...in · xi11 · . . . · xinn = λk · m(x1 , . . . , xn ).
Close
Quit
Objective of Science . . .
11.
Proof (Part 2)
Partial Orders . . .
• Reminder: f (x1 , . . . , xn ) is a sum of monomials m(x1 , . . . , xn ) = ai1 ...in ·
xi11
· ... ·
xinn .
Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . .
• For each monomial, by its order, we will understand the sum k = i1 + . . . + in . • For each order k, there are finitely many possible combinations of integers i1 , . . . , in for which i1 +. . .+in = k. • So, there are finitely many possible monomials of the order k. • Let Pk (x1 , . . . , xn ) denote the sum of all the monomials of order k in the expansion of f (x1 , . . . , xn ). • Then, we have f (x1 , . . . , xn ) = P0 +P1 (x1 , . . . , xn )+P2 (x1 , x2 , . . . , xn )+. . .
Towards an . . . What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 12 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
12.
Proof (Part 3)
• f (x) = P0 +P1 (x1 , . . . , xn )+P2 (x1 , . . . , xn )+. . . , where Pk (x1 , . . . , xn ) is the sum of monomials of order k. • Some of the sums Pk may be zeros – if the expansion of f has no monomials of the corresponding order. • Let k0 be the first index for which the term Pk0 (x1 , . . . , xn ) is not identically 0. Then,
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
f (x1 , . . . , xn ) = Pk0 (x1 , . . . , xn )+Pk0 +1 (x1 , . . . , xn )+. . . • Since the family F is scale-invariant, it also contains fλ (x1 , . . . , xn ) = f (λ · x1 , . . . , λ · xn ). • At this re-scaling, each term Pk is multiplied by λk . • Thus, we get fλ (x) = λk0 ·Pk0 (x1 , . . . , xn )+λk0 +1 ·Pk0 +1 (x1 , . . . , xn )+. . .
JJ
II
J
I
Page 13 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
13.
Proof (Part 4)
• Proven: fλ (x) = λk0 · Pk0 (x) + λk0 +1 · Pk0 +1 (x) + . . . ∈ F. • Since F is a linear space, it also contains a function λ−k0 · fλ (x) = Pk0 (x) + λ · Pk0 +1 (x) + . . .
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . . Towards an . . .
• Since F is finite-dimensional, it is closed under turning to a limit.
What If . . . Acknowledgments
• In the limit λ → 0, we conclude that the term Pk0 (x) also belongs to the family F : Pk0 (x) ∈ F .
JJ
II
• Since F is a linear space, this means that the difference
J
I
f (x) − Pk0 (x) = Pk0 +1 (x) + Pk0 +2 (x) + . . . ∈ F. • Let k1 be the first index k1 > k0 for which the term Pk1 (x) is not identically 0. • Then we can similarly conclude that the term Pk1 (x) also belongs to the family F , etc.
Title Page
Page 14 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
14.
Proof (Conclusion)
• We can therefore conclude that: – for every index k for which Pk (x) 6≡ 0, – this term Pk (x) also belongs to the family F . • Fact: monomials of different total order are linearly independent: – if there were infinitely many non-zero terms Pk in the expansion of the function f (x), – we would have infinitely many linearly independent function in the family F – which contradicts to our assumption that the family F is a finite-dimensional linear space. • So, there are only finitely many non-zero Pk . • Hence, f (x) is a sum of finitely many monomials – i.e., a polynomial.
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 15 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
15.
Towards an Alternative Justification Based on Optimality
• Idea: we would like to select the optimal finite-dimensional family of analytical functions F .
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . .
• What is an optimality criterion: when we can decide – whether F is better than F 0 (denoted F 0 ≺ F ) – or F 0 is better than F (F ≺ F 0 )
What If . . . Acknowledgments
– or F 0 is of the same quality as F (denoted F ≡ F 0 ). 0
Towards an . . .
0
• E.g.: numerical criterion F ≺ F ⇔ J(F ) < J(F ). • More general case: – when J(F ) = J(F 0 ), e.g., for average approximation accuracy J(F ), – we can still choose between F and F 0 based on some other criteria J 0 (e.g., computational simplicity).
Title Page
JJ
II
J
I
Page 16 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
16.
Towards General Description of Optimality
Partial Orders . . . Numerical . . .
• Reminder:
Need to Combine . . .
0
– when J(F ) = J(F ), e.g., for average approximation accuracy J(F ),
Mathematical . . . Towards a General . . .
– we can still choose between F and F 0 based on some other criteria J 0 (e.g., computational simplicity).
Towards an . . . What If . . . Acknowledgments
• The resulting criterion is non-numerical:
Title Page
0
0
0
0
0
0
F ≺ F ⇔ J(F ) < J(F )∨(J(F ) = J(F ) & J (F ) < J (F ). • General definition: a (pre)-ordering relation . • Natural requirement: which operation is better should be not depend on the choice of measuring unit: F ≺ F 0 ⇔ Fλ ≺ Fλ0 , where Fλ = {fλ : f ∈ F }.
JJ
II
J
I
Page 17 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
17.
Optimization Approach: Definitions
• We consider the set A of all finite-dimensional spaces of analytical functions.
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . .
• By an optimality criterion, we mean a pre-ordering (i.e., a transitive, reflexive relation) on the set A. • An optimality criterion on the class of all finitedimensional is called scale-invariant if – for all F , F 0 , and λ 6= 0, – F F 0 implies Fλ Fλ0 . • An optimality criterion is called final if there exists
Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 18 of 25
– one and only one space F Go Back
– that is preferable to all the others, i.e., for which F 0 F for all F 0 6= F .
Full Screen
Close
Quit
Objective of Science . . .
18.
Why Final Criterion: Motivations
• Reminder: an optimality criterion is final if there exists one and only one optimal space F .
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . .
• If no space is optimal relative to some criterion, then this criterion is useless. • If the criterion selects several spaces F as equally good, then we can also optimize something else.
Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
• Example: 0
– if F and F have the same average approximation accuracy, – we can select, among them, the one which is easier to compute. • Thus, such criteria can be adjusted. • So, for the final criterion, the optimal space is unique.
JJ
II
J
I
Page 19 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
19.
Optimization Approach: Main Result
• Condition: Fopt is optimal w.r.t. some scale-invariant and final optimality criterion.
Partial Orders . . . Numerical . . . Need to Combine . . . Mathematical . . .
• Conclusion: all elements of Fopt are polynomials.
Towards a General . . .
• Proof:
Towards an . . .
– optimality means F Fopt for all F ∈ A; – in particular, Fλ−1 Fopt for all F ∈ A; – due to scale-invariance of , we have F (Fopt )λ for all F ∈ A; – thus, (Fopt )λ is optimal; – since there is only one optimal space, we have (Fopt )λ = Fopt ; – thus, the space Fopt is scale-invariant; – we already know that in this case, all f ∈ Fopt are polynomials.
What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 20 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
20.
What If f (x1 , . . . , xn ) Is Only Smooth?
Partial Orders . . . Numerical . . .
Definition. Let n be an arbitrary integer. We say that a finite-dimensional linear space F of smooth functions of n variables is affine-invariant if for every f ∈ F and for every linear transormation T : Rn → Rn , the function
Need to Combine . . . Mathematical . . . Towards a General . . . Towards an . . .
def
fT (x) = f (T x) also belongs to the family F . Main result. For every affine-invariant finite-dimensional linear space F of smooth functions, every element f ∈ F is a polynomial.
What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 21 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
21.
Proof: Main Ideas
Partial Orders . . .
• Let f1 (x), . . . , fm (x) be the basis of F . • For every i ≤ m, for every variable xj and for every λ > 0, we have fi (x1 , . . . , xj−1 , λ · xj , xj+1 , . . . , xn ) ∈ F.
Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . . Towards an . . . What If . . .
• Since fi form a basis, for some cik (λ), we have
Acknowledgments Title Page
fi (x1 , . . . , xj−1 , λ · xj , xj+1 , . . . , xn ) = m X
cik (λ) · fk (x1 , . . . , xj−1 , xj , xj+1 , . . . , xn ).
k=1
JJ
II
J
I
Page 22 of 25
• Differentiating both sides by λ, we get
Go Back
m
X ∂fi xj · = cjk · fk . ∂xj k=1
Full Screen
Close
Quit
Objective of Science . . .
22.
Proof (cont-d)
Partial Orders . . . Numerical . . .
m
X ∂fi • Reminder: xj · = cjk · fk . ∂xj
Need to Combine . . .
k=1
Mathematical . . .
m
X ∂fi = cik · fk . • For Xj = ln(xj ), we have ∂Xj def
k=1
• In terms of Xj , we have a system of linear ODEs with constant coefficients. • A general solution to such a system is a linear combination of terms xαj
• exp(α · Xj ) = (with possible complex α) and p • Xj · exp(α · Xj ) = xαj · lnp (xj ). • A general linear transformation leads to different terms – except when we have xαj for integer α ≥ 0. • Thus, every f ∈ F is a polynomial in each variable – hence a polynomial.
Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 23 of 25
Go Back
Full Screen
Close
Quit
Objective of Science . . .
23.
Acknowledgments
This work was supported in part
Partial Orders . . . Numerical . . . Need to Combine . . .
• by the National Science Foundation grants HRD-0734825 and DUE-0926721, • by Grant 1 T36 GM078000-01 from the National Institutes of Health, ˇ • by Grant MSM 6198898701 from MSMT of Czech Republic, • by Grant 5015 “Application of fuzzy logic with operators in the knowledge based systems” from the Science and Technology Centre in Ukraine (STCU), funded by European Union, and • by the International Finsler Foundation.
Mathematical . . . Towards a General . . . Towards an . . . What If . . . Acknowledgments Title Page
JJ
II
J
I
Page 24 of 25
Go Back
Full Screen
Close
Quit
24.
References Related to Other Application Areas
Objective of Science . . . Partial Orders . . .
• Luce, R. D., Raiffa, R.: Games and decisions: introduction and critical survey, Dover, New York, 1989. • Nguyen H. T., Kosheleva O., Kreinovich, V.: “Decision Making Beyond Arrow’s Impossibility Theorem”, International Journal of Intelligent Systems, 24(1), 27– 47 (2009).
Numerical . . . Need to Combine . . . Mathematical . . . Towards a General . . . Towards an . . . What If . . . Acknowledgments
Title Page
• Nguyen, H. T., Walker, E. A.: A First Course in Fuzzy Logic, Chapman & Hall/CRC Press, Boca Raton, Florida, 2006.
JJ
II
J
I
Page 25 of 25 Go Back Full Screen Close Quit
