ON THE CONSTRUCTION OF CONVERGING HIERARCHIES FOR POLYNOMIAL OPTIMIZATION BASED ON CERTIFICATES OF GLOBAL POSITIVITY∗ AMIR ALI AHMADI†
Abstract.
AND
GEORGINA HALL‡
In recent years, techniques based on convex optimization and real algebra that
produce converging hierarchies of lower bounds for polynomial optimization problems (POPs) have gained much popularity. At their heart, these hierarchies rely crucially on Positivstellensätze from the late 20th century (e.g., due to Stengle, Putinar, or Schmüdgen) that certify positivity of a polynomial on an arbitrary basic semialgebraic set.
In this paper, we show that such hierarchies
could in fact be designed from much more limited Positivstellensätze dating back to the early 20th century that only certify positivity of a polynomial globally. More precisely, we show that any inner approximation to the cone of positive homogeneous polynomials that is arbitrarily tight can be turned into a converging hierarchy for general polynomial optimization problems with compact feasible sets. This in particular leads to a semide nite programming-based hierarchy that relies solely on Artin's solution to Hilbert's 17th problem.
We also use a classical result of Polyá on global positivity of
even forms to construct an optimization-free converging hierarchy for general POPs with compact feasible sets. This hierarchy only requires polynomial multiplication and checking nonnegativity of coe cients of certain xed polynomials.
As a corollary, we obtain new linear programming and
second-order cone programming-based hierarchies for POPs that rely on the recently introduced concepts of dsos (diagonally dominant sum of squares) and sdsos (scaled diagonally dominant sum of squares) polynomials.
Key words.
Positivstellensätze, polynomial optimization, convex optimization
AMS subject classi cations.
1. Introduction.
14Q99, 90C05, 90C26, 90C22, 90-03
A polynomial optimization problem (POP) is an optimization
problem of the form
inf
x∈Rn
(1)
s.t. where
p, gi , i = 1, . . . , m,
and with real coe cients.
p(x) gi (x) ≥ 0, i = 1, . . . , m,
are polynomial functions in
n
variables
x := (x1 , . . . , xn )
It is well-known that polynomial optimization is a hard
problem to solve in general. For example, simply testing whether the optimal value of problem (1) is smaller than or equal to some rational number
k
is NP-hard already
when the objective is quadratic and the constraints are linear [14]. Nevertheless, these problems remain topical due to their numerous applications throughout engineering, operations research, and applied mathematics (see, e.g., [10, 6, 2]). In this paper, we are interested in obtaining lower bounds on the optimal value of problem (1). We focus on a class of methods which construct hierarchies of tractable convex optimization problems whose optimal values are lowerbounds on the optimal value of (1), with convergence to it as the sequence progresses.
This implies that even though the
original POP is nonconvex, one can obtain increasingly accurate lower bounds on its optimal value by solving convex optimization problems. One method for constructing ∗ This work is partially funded by the DARPA Young Faculty Award, the Young Investigator Award of the AFOSR, the CAREER Award of the NSF, the Google Faculty Award, and the Sloan Fellowship.
† Department
of
Operations
Research
and
Financial
Engineering,
Princeton,
NJ
08540
Engineering,
Princeton,
NJ
08540
(
[email protected], http://aaa.princeton.edu).
‡ Department
of
Operations
Research
and
Financial
(
[email protected], http://scholar.princeton.edu/ghall). 1
2 these hierarchies of optimization problems that has gained attention in recent years
Positivstellensätze
relies on the use of
(see, e.g., [11] for a survey). Positivstellensätze
are algebraic identities that certify infeasibility of a set of polynomial inequalities, or
1
equivalently , positivity of a polynomial on a basic semialgebraic set.
(Recall that
a basic semialgebraic set is a set de ned by nitely many polynomial inequalities.) These Positivstellensätze can be used to prove lowerbounds on POPs. Indeed, if we denote the feasible set of (1) by
S,
sup
the optimal value of problem (1) is equivalent to
γ
γ
(2)
p(x) − γ ≥ 0, ∀x ∈ S.
s.t. Hence if
γ
is a strict lower bound on (1), we have that
p(x) − γ > 0
on
S,
a fact
that can be certi ed using Positivstellensätze. At a conceptual level, hierarchies that provide lower bounds on (1) are constructed thus: we x the size of the certi cate at each level of the hierarchy and search for the largest at hand can certify positivity of
p(x) − γ
over
S
γ
such that the Positivstellensätze
with a certi cate of this size. As the
sequence progresses, we increase the size of the certi cates allowed, hence obtaining increasingly accurate lower bounds on (1). Below, we present three of the better-known Positivstellensätze, given respectively by Stengle [25], Schmüdgen [21], and Putinar [19]. These all rely on sum of squares certi cates. We recall that a polynomial is a
sum of squares
(sos) if it can be written
as a sum of squares of other polynomials. We start with Stengle's Positivstellensatz, which certi es infeasibility of a set of polynomial inequalities. It is sometimes referred to as the Positivstellensatz in related literature as it requires no assumptions, contrarily to Schmüdgen and Putinar's theorems which can be viewed as re nements of Stengle's result under additional assumptions.
Theorem 1.1
(Stengle's Positivstellensatz [25]).
The basic semialgebraic set
S = {x ∈ Rn | g1 (x) ≥ 0, . . . , gm (x) ≥ 0}
is empty if and only if there exist sum of squares polynomials s0 (x),s1 (x),. . ., sm (x), s12 (x), s13 (x),. . ., s123...m (x) such that −1 = s0 (x) +
X
X
si (x)gi (x) +
i
sij (x)gi (x)gj (x) + . . . + s123...m (x)g1 (x) . . . gm (x).
{i,j}
The next two theorems, due to Schmüdgen and Putinar, certify positivity of a polynomial
p
over a basic semialgebraic set
S.
They impose additional compactness as-
sumptions comparatively to Stengle's Positivstellensatz.
Theorem 1.2
(Schmüdgen's Positivstellensatz [21]).
Assume that the set
S = {x ∈ Rn | g1 (x) ≥ 0, . . . , gm (x) ≥ 0}
is compact. If a polynomial p is positive on S , then p(x) = s0 (x) +
X
si (x)gi (x) +
i
X
sij (x)gi (x)gj (x) + . . . + s123...m (x)g1 (x) . . . gm (x),
{i,j}
where s0 (x),s1 (x),. . ., sm (x), s12 (x), s13 (x),. . ., s123...m (x) are sums of squares. 1 Note the set
{x ∈ Rn | g1 (x) ≥ 0, . . . , gm (x) ≥ 0} | g2 (x) ≥ 0, . . . , gm (x) ≥ 0}.
that the set
{x ∈
Rn
is empty if and only if
−g1 (x) > 0
on
3
CONVERGING HIERARCHIES FOR POLYNOMIAL OPTIMIZATION
Theorem 1.3
Let
(Putinar's Positivstellensatz [19]).
S = {x ∈ Rn | g1 (x) ≥ 0, . . . , gm (x) ≥ 0}
and assume that {g1 , . . . , gm } satisfy the Archimedean property, i.e., there exists N ∈ N such that N−
X
x2i = σ0 (x) + σ1 (x)g1 (x) + . . . + σm (x)gm (x),
i
where σ1 (x), . . . , σm (x) are sums of squares. If a polynomial p is positive on S , then p(x) = s0 (x) + s1 (x)g1 (x) + . . . + sm (x)gm (x),
where s1 (x), . . . , sm (x) are sums of squares. Note that these three Positivstellensätze involve in their expressions sum of squares polynomials of unspeci ed degree.
To construct hierarchies of tractable optimiza-
tion problems for (2), we x this degree: such that positivity of
p(x) − γ
over
S
at level
r,
we search for the largest
γ
can be certi ed using the Positivstellensätze
where the degrees of all sos polynomials are taken to be less than or equal to
2r.
Solving each level of these hierarchies is then a semide nite program (SDP). This is a consequence of the fact that one can optimize over (or test membership to) the set of sum of squares polynomials of xed degree using semide nite programming [16, 15, 9]. Indeed, a polynomial
p
of degree
2d
and in
n
variables is a sum of squares
p(x) = z(x)T Qz(x), d T where z(x) = (1, x1 , . . . , xn , . . . , xn ) is the standard vector of monomials in n variables and of degree less than or equal to d. We remark that the hierarchy obtained
if and only if there exists a symmetric matrix
Q0
such that
from Stengle's Positivstellensatz was proposed and analyzed by Parrilo in [16]; the hierarchy obtained from Putinar's Positivstellensatz was proposed and analyzed by Lasserre in [9]. There have been more recent works that provide constructive proofs of Schmüdgen and Putinar's Positivstellensätze; see [5, 22, 24]. These proofs rely on other Positivstellensätze, e.g., a result by Polyá (see Theorem 1.6 below) in [22, 24], and the same result by Polyá, Farkas' lemma, and Stengle's Positivstellensatz in [5]. There has further been an eort to derive complexity bounds for Schmüdgen and Putinar's Positivstellensätze in recent years; see [13, 23]. On a historical note, Stengle, Schmüdgen, and Putinar's Positivstellensätze were derived in the latter half of the 20th century.
As mentioned previously, they all
certify positivity of a polynomial over an arbitrary basic semialgebraic set (modulo compactness assumptions). By contrast, there are Positivstellensätze from the early 20th century that certify positivity of a polynomial
globally.
Perhaps the most well-
known Positivstellensatz of this type is due to Artin in 1927, in response to Hilbert's 17th problem. Artin shows that any nonnegative polynomial is a sum of squares of rational functions. Here is an equivalent formulation of this statement:
Theorem 1.4 (Artin [4]). For any nonnegative polynomial p, there exists an sos polynomial q such that p · q is a sum of squares. To the best of our knowledge, in this area, all converging hierarchies of lower bounds for POPs are based o of Positivstellensätze that certify nonnegativity of a polynomial over an arbitrary basic semialgebraic set. In this paper, we show that in fact, under compactness assumptions, it su ces to have only global certi cates of nonnegativity (such as the one given by Artin) to produce a converging hierarchy for general POPs.
4 As a matter of fact, even weaker statements that apply only to globally positive (as opposed to globally nonnegative) forms are enough to derive converging hierarchies for POPs. Examples of such statements are due to Habicht [8] and Reznick [20]. With such an additional positivity assumption, more can usually be said about the structure of the polynomial
q
in Artin's result. Below, we present the result by Reznick.
Theorem 1.5P (Reznick [20]).
such that p(x) · (
For any positive de nite form p, there exists r ∈ N
2 r i xi ) is a sum of squares.
We show in this paper that this Positivstellensatz also gives rise to a converging hierarchy for POPs with a compact feasible set similarly to the one generated by Artin's Positivstellensatz. Through their connections to sums of squares, the two hierarchies obtained using the theorems of Reznick and Artin are semide nite programming-based. In this paper, we also derive an optimization-free converging hierarchy for POPs with compact feasible sets where each level of the hierarchy only requires that we be able to test nonnegativity of the coe cients of a given xed polynomial.
To the best of our
knowledge, this is the rst converging hierarchy of lower bounds for POPs which does not require that convex optimization problems be solved at each of its levels. To construct this hierarchy, we use a result of Polyá [17], which just like Artin's and Reznick's Positivstellensätze, certi es global positivity of forms. However this result is restricted to even forms. Recall that a form
p is even
if each of the variables featuring
in its individual monomials has an even power. This is equivalent (see [7, Lemma 2]) to
p
being invariant under change of sign of each of its coordinates, i.e.,
p(x1 , . . . , xn ) = p(−x1 , . . . , xn ) = · · · = p(x1 , . . . , −xn ).
Theorem 1.6 (Polyá P [17]). For any positive de nite even form p, there exists r ∈ N such that p(x) · ( i x2i )r has nonnegative coe cients.2 Our aforementioned hierarchy enables us to obtain faster-converging linear programming (LP) and second-order cone programming (SOCP)-based hierarchies for general POPs with compact feasible sets that rely on the concepts of
dsos and sdsos polynomi-
als. These are recently introduced inner approximations to the set of sos polynomials that have shown much better scalability properties in practice [3]. As a nal remark, we wish to stress the point that the goal of this paper is rst and foremost theoretical, i.e., to provide methods for constructing converging hierarchies of lower bounds for POPs using as sole building blocks certi cates of global positivity. We do not make any claims that these hierarchies can outperform the popular existing hierarchies due, e.g., to Lasserre [9] and Parrilo [16].
We do
believe however that the optimization-free hierarchy presented in Section 4.1 could potentially be of interest in large-scale applications where the convex optimization problems appearing in traditional hierarchies are too cumbersome to solve.
1.1. Outline of the paper.
The paper is structured as follows. In Section 2, we
show that if one can inner approximate the cone of positive de nite forms arbitrarily well (with certain basic properties), then one can produce a converging hierarchy of lower bounds for POPs with compact feasible sets (Theorem 2.4).
2A form
h
This relies
perhaps better-known but equivalent formulation of this theorem is the following: for any that is positive on the standard simplex, there exists
r ∈ N
such that
nonnegative coe cients. The two formulations are equivalent by simply letting
P h(x) · ( i xi )r p(x) = h(x2 ).
has
5
CONVERGING HIERARCHIES FOR POLYNOMIAL OPTIMIZATION
on a reduction (Theorem 2.1) that reduces the problem of certifying a strict lower bound on a POP to that of proving positivity of a certain form.
In Section 3, we
see how this result can be used to derive semide nite programming-based converging hierarchies (Theorems 3.2 and 3.4) from the Positivstellensätze by Artin (Theorem 1.4) and Reznick (Theorem 1.5). In Section 4, we derive an optimization-free hierarchy (Theorem 4.1) from the Positivstellensatz of Polyá (Theorem 1.6) as well as LP and SOCP-based hierarchies which rely on dsos/sdsos polynomials (Corollary 4.8).
We
conclude with a few open problems in Section 5.
1.2. Notation and basic de nitions. to denote that a symmetric matrix
A
We use the standard notation
is positive semide nite. Recall that a
A 0
form
is
a homogeneous polynomial, i.e., a polynomial whose monomials all have the same degree.
We denote the degree of a form
f
by
deg(f ).
We say that a form
f
is
nonnegative (or positive semide nite) if f (x) ≥ 0, for all x ∈ Rn (we write f ≥ 0). A n form f is positive de nite (pd) if f (x) > 0, for all nonzero x in R (we write f > 0). Throughout the paper, we denote the set of forms (resp. forms) in
n
variables and of degree
d
Hn,d B(0, R)
by
(resp
Pn,d ).
the set of nonnegative We denote the ball of
R and centered at the origin by and the unit sphere in x-space, i.e., {x ∈ Rn | ||x||2 = 1}, by Sx . We use the shorthand f (y 2 − z 2 ) for y, z ∈ Rn to 2 2 2 2 denote f (y1 − z1 , . . . , yn − zn ). We say that a scalar γ is a strict lower bound on (1) if p(x) > γ, ∀x ∈ S . Finally, we ask the reader to carefully read Remark 2.3 which radius
contains the details of a notational overwriting occurring before Theorem 2.4 and valid from then on throughout the paper.
This overwriting makes the paper much
simpler to parse.
2. Constructing converging hierarchies for POP using global certi cates of positivity. Consider the polynomial optimization problem in (1) and denote its optimal value by
p∗ .
Let
d
2d is the smallest even integer larger p, gi , i = 1, . . . , m. We denote the feasible
be such that
than or equal to the maximum degree of set of our optimization problem by
S = {x ∈ Rn | gi (x) ≥ 0, i = 1, . . . , m} R. From this, it is easy to gi (x) over the set S : as S is contained in a ball of radius R, we have |xi | ≤ R, for all i = 1, . . . , n. We then use this to upper bound each monomial in gi and consequently gi itself. We use the notation ηi to denote these upper bounds, i.e., gi (x) ≤ ηi , for all i = 1, . . . , m and for all x ∈ S . Similarly, we can provide an upperbound on −p(x). We denote such a bound by β , i.e., −p(x) ≤ β, ∀x ∈ S.
and assume that
S
is contained within a ball of radius
provide (possibly very loose) upper bounds on
The goal of this section is to produce a method for constructing converging hierarchies of lower bounds for POPs if we have access to arbitrarily accurate inner approximations of the set of positive de nite forms. The rst theorem (Theorem 2.1) connects lower bounds on (1) to positive de niteness of a related form. The second theorem (Theorem 2.4) shows how this can be used to derive a hierarchy for POPs.
Theorem 2.1. Consider the general polynomial optimization problem in (1) and recall that d is such that 2d is the smallest even integer larger than or equal to the maximum degree of p, gi , i = 1, . . . , m. Suppose S ⊆ B(0, R) for some positive scalar R. Let ηi , i = 1, . . . , m (resp. β ) be any nite upper bounds on gi (x), i = 1, . . . , m (resp. −p(x)). Then, a scalar γ is a strict lower bound on (1) if and only if the homogeneous
6
sum of squares polynomial (3)
fγ (x, s, y) := γy
2d
2d
− y p(x/y) −
2 s20 y 2d−2
+
m X
y 2d gi (x/y) − s2i y 2d−2
2
i=1
+
(R +
m X
ηi + β + γ)d y 2d − (
i=1
n X
x2i +
i=1
m X
!2 s2i )d − s2d m+1
i=0
of degree 4d and in n + m + 3 variables (x1 , . . . , xn , s0 , . . . , sm , sm+1 , y) is positive de nite. Proof. It is easy to see that γ is a strict lower bound on (1) if and only if the set T := {x ∈ Rn | γ − p(x) ≥ 0;
gi (x) ≥ 0, i = 1, . . . , m;
X
x2i ≤ R}
i is empty. Indeed, if This implies that
γ
the intersection of
T
is nonempty, then there exists a point
x ˇ∈S
such that
p(ˇ x) ≤ γ .
T
is empty,
cannot be a strict lower bound on (1). Conversely, if
S
{x | γ − p(x) ≥ 0}
with
is empty, which implies that
∀x ∈ S ,
p(x) > γ . We now de ne the set:
(4)
Ts = {(x, s) ∈ Rn+m+2 | γ − p(x) = s20 ; gi (x) = s2i , i = 1, . . . , m; m n m X X X d 2 (R + ηi + β + γ) − ( xi + s2i )d − s2d m+1 = 0}. i=1
i=1
i=0
Ts is empty if and only if T is empty. Indeed, if Ts is nonempty, then there exists x ˆ ∈ Rn and sˆ ∈ Rm+2 such that the three sets of equations are satis ed. This obviously implies that γ − p(ˆ x) ≥ 0 and that gi (ˆ x) ≥ 0, for all i = 1, . . . , m. It further P 2 implies that x ˆ ≤ R as by assumption, if x ˆ ∈ S , then x ˆ is in a ball of radius R. i i Conversely, suppose now that T is nonempty. There exists x ˜ such that γ − p(˜ x) ≥ 0, P gi (˜ x) ≥ 0 for i = 1, . . . , m, and i x˜i 2 ≤ R. Hence, there exist s˜0 , . . . , s˜m such that
Note that
γ − p(˜ x) = s˜0 2 and gi (˜ x) = s˜i 2 , i = 1, . . . , m. P 2 Combining the fact that ˜i ≤ R and the fact that ηi , i = 1, . . . , m ix are upperbounds on gi (resp. γ − p(˜ x)), we obtain: R+
m X
ηi + β + γ ≥
i=1
n X
x˜i 2 +
i=1
m X
(resp.
γ + β)
s˜i 2 .
i=0
d, we show the existence of s˜m+1 . Ts is empty if and only if fγ (x, s, y) is positive de nite. Suppose that Ts is nonempty, i.e., there exists (˘ x, s˘) ∈ Rn+m+2 such that the equalities given in (4) hold. Note then that fγ (˘ x, s˘, 1) = 0. As (˘ x, s˘, 1) is nonzero, this implies that fγ (˘ x, s˘, y˘) is not positive de nite. For the converse, assume that fγ (x, s, y) is not positive de nite. As fγ (x, s, y) is a sum of squares and hence nonnegative, this means that there exists nonzero (¯ x, s¯, y¯) such that f (¯ x, s¯, y¯) = 0. We proceed in two cases. If y¯ 6= 0, it is easy to see that (¯ x/¯ y , s¯/¯ y ) ∈ Ts and Ts is nonempty. Consider now the case where y¯ = 0. The third square in fγ being equal to zero gives us: By raising both sides of the inequality to the power We now show that
m X X 2 −( x ¯i + s¯2i )d = s¯2d m+1 . i
i=0
CONVERGING HIERARCHIES FOR POLYNOMIAL OPTIMIZATION
This implies that
s¯m+1 = 0 and that x ¯1 = . . . = x ¯m = s¯0 = . . . = s¯m = 0 (¯ x, s¯, y¯) is nonzero.
7 which
contradicts the fact that
Remark 2.2. Note that Theorem 2.1 implies that testing feasibility of a set of polynomial inequalities is no harder than checking whether a homogeneous polynomial that is sos has a zero. Indeed, as mentioned before, the basic semialgebraic set {x | g1 (x) ≥ 0, . . . , gm (x) ≥ 0}
is empty if and only if γ = 0 is a strict lower bound on the POP inf
− g1 (x)
s.t.
g2 (x) ≥ 0, . . . , gm (x) ≥ 0.
x
In principle, this reduction can open up new possibilities for algorithms for testing feasibility of a basic semialgebraic set. For example, the work in [1] shows that positive de niteness of a form f is equivalent to global asymptotic stability of the polynomial vector eld x˙ = −∇f (x). One could as a consequence search for Lyapunov functions, as is done in [1, Example 2.1.], to certify positivity of forms. Conversely, simulating trajectories of the above vector eld can be used to minimize f and potentially nd its nontrivial zeros, which, by our reduction, can be turned into a point that belongs to the basic semialgebraic set at hand. We further remark that one can always take the degree of the sos form fγ in (3) whose positivity is under consideration to be equal to four. This can be done by changing the general POP in (1) to only have quadratic constraints and a quadratic objective via an iterative introduction of new variables and new constraints in the following fashion: xij = xi xj .
Remark 2.3 (Notational remark). As a consequence of Theorem 2.1, we now know that certifying lower bounds on (1) is equivalent to proving positivity of the form fγ that appears in (3). To simplify notation, we take this form to have n variables and be of degree 2d from now on (except for our Positivstellensätze in Corollaries 3.5 and 4.5 which stand on their own). To connect back to problem (1) and the original notation, the reader should replace every occurrence of n and d in the future as follows: n ← n + m + 3,
d ← 2d.
Recall that n was previously the dimension of the decision variable of problem (1), d was such that 2d is the smallest even integer larger than or equal to the maximum degree of gi and p in (1), and m was the number of constraints of problem (1). Our next theorem shows that, modulo some technical assumptions, if one can inner approximate the set of positive de nite forms arbitrarily well (conditions (a) and (b)), then one can construct a converging hierarchy for POPs.
Theorem 2.4.
r Let Kn,2d be a sequence of sets (indexed by r) of homogeneous polynomials in n variables and of degree 2d with the following properties: r 0 (a) Kn,2d ⊆ Pn,2d , ∀r, and there exists a pd form sn,2d ∈ Kn,2d . r (b) If p > 0, then ∃r ∈ N such that p ∈ Kn,2d . r+1 r (c) Kn,2d ⊆ Kn,2d , ∀r. r r (d) If p ∈ Kn,2d , then ∀ ∈ [0, 1], p + sn,d ∈ Kn,2d . Recall the de nition of fγ (z) given in (3). Consider the hierarchy of optimization
8
problems indexed by r: lr := sup
γ
γ
(5)
s.t.
1 r . fγ (z) − sn,2d (z) ∈ Kn,2d r
Then, lr ≤ p∗ for all r, {lr } is nondecreasing, and limr→∞ lr = p∗ . Proof. We rst show that the sequence {lr } is upperbounded by p∗ . a scalar
γ
Suppose that
satis es
1 r fγ (z) − sn,2d (z) ∈ Kn,2d . r 1 1 We then have fγ (z)− sn,2d (z) ∈ Pn,2d using (a). This implies that fγ (z) ≥ sn,2d (z), r r and hence fγ is pd as sn,2d is pd. From Theorem 2.1, it follows that γ has to be a ∗ ∗ strict lower bound on (1). As any such γ satis es γ < p , we have that lr ≤ p for all r . We now show monotonicity of the sequence
{lr }.
Let
γ
be such that
1 r . fγ (z) − sn,2d (z) ∈ Kn,2d r We have the following identity:
fγ (z) −
1 1 1 sn,2d (z) = fγ (z) − sn,2d (z) + sn,2d (z). r+1 r r(r + 1)
Now, using the assumption and properties (c) and (d), we conclude that
fγ (z) −
1 r+1 sn,2d (z) ∈ Kn,2d . r+1
r+1 1 r } ⊆ {γ | fγ (z) − r+1 sn,2d (z) ∈ Kn,2d } {γ | fγ (z) − 1r sn,2d (z) ∈ Kn,2d lr ≤ lr+1 .
This implies that and that
Note that as the sequence
{lr }
is upperbounded and nondecreasing, it converges.
Let us show that the limit of this sequence is
p∗ .
To do this, we show that for any strict
r such that fγ (z) − 1r sn,2d (z) ∈ γ is a strict lower bound, fγ (z) is positive de nite. Hence, 1 0 by continuity, there exists a positive integer r such that fγ (z) − 0 sn,2d (z) is positive r 00 de nite. Using (b), this implies that there exists a positive integer r such that lower bound
γ
on (1), there exists a positive integer
r Kn,2d . By Theorem 2.1, as
fγ (z) −
(6)
We now proceed in two cases. If conclude. If
0
r ≤r
00
r = r00
r00 ≤ r0 ,
we take
r = r0
and use property (c) to
, we have
fγ (z) − We take
1 r 00 sn,2d (z) ∈ Kn,2d . r0
1 1 r00 − r0 s (z) = f (z) − s (z) + sn,2d (z). n,2d γ n,2d r00 r0 r0 · r00
and use (6) and properties (c) and (d) to conclude.
Remark 2.5. Note that condition (d) is subsumed by the more natural condition r that Kn,d be a convex cone for any n, d, and r. However, there are interesting and relevant cones which we cannot prove to be convex though they trivially satisfy condition (d) (see Theorem 3.2 for an example).
9
CONVERGING HIERARCHIES FOR POLYNOMIAL OPTIMIZATION
3. Semide nite programming-based hierarchies obtained from Artin's and Reznick's Positivstellensätze. In this section, we construct two dierent semide nite programming-based hierarchies for POPs using Positivstellensätze derived by Artin (Theorem 1.4) and Reznick (Theorem 1.5). To do this, we introduce two sets of cones that we call the Artin and Reznick cones.
Definition 3.1.
We de ne the
Reznick cone
r Rn,2d := {p ∈ Hn,2d | p(x) ·
of level r to be
n X
!r x2i
is sos}.
i=1
Similarly, we de ne the
Artin cone
of level r to be
Arn,2d := {p ∈ Hn,2d | p(x) · q(x) is sos for some sos form q of degree 2r}. We show that both of these cones produce hierarchies of the type discussed in Theorem 2.4. Recall that
p∗
is the optimal value of problem (1) and that
fγ
is de ned
as in (3) with the change of notation discussed in Remark 2.3.
Theorem 3.2.
Consider the hierarchy of optimization problems indexed by r: lr := sup
γ
γ (7)
n 1 X 2 d r fγ (z) − ( z ) ∈ Rn,2d . r i=1 i
s.t.
Then, lr ≤ p∗ for all r, {lr } is nondecreasing, and limr→∞ lr = p∗ . Proof.
It su ces to show that the Reznick cones
in Theorem 2.4.
r Rn,2d
satisfy properties (a)-(d)
The result will then follow from that theorem.
For property (a), P p(x) · ( i x2i )r is a sum of squares and hence r nonnegative, p(x) must be nonnegative, so Rn,2d ⊆ Pn,2d . Furthermore, the form P 2 d 0 sn,2d := ( i xi ) belongs to Rn,2d and is positive de nite. Property (b) is veri ed P 2 r as a consequence of Theorem 1.5. For (c), note that if p(x) · ( i xi ) is sos, then P 2 r+1 p(x) · ( i xi ) is sos since the product of two sos polynomials is sos. Finally, for r property (d), note that Rn,2d is a convex cone. Indeed, for any λ ∈ [0, 1],
it is clear that, as
P ( i x2i )r > 0
(λp(x) + (1 − λ)q(x)) · (
X
and
X X x2i )r = λp(x)( x2i )r + (1 − λ)q(x)( x2i )r
i is sos if
p
fact that
r q are in Rn,2d . Combining the 2 d r x ) ∈ R , we obtain (d). n,d i i
and
(
i fact that
i r Rn,2d
is a convex cone and the
P
Remark 3.3. To solve a xed level r of the hierarchy given in Theorem 3.2, one must proceed by bisection on γ. Bisection here would produce a sequence of upper bounds {Uk } and lower bounds {Lk } on lr as follows. At iteration k, we test whether k k γ = Uk +L is feasible for (7). If it is, then we take Lk+1 = Uk +L and Uk+1 = Uk . 2 2 Uk +Lk If it is not, we take Uk+1 = 2 and Lk+1 = Lk . We stop when |Uk − Lk | < , where is a prescribed accuracy, and the algorithm returns lr, = Lk . Note that lr − ≤ lr, ≤ lr and that to obtain lr, , one needs to take a logarithmic (in 1 ) number of steps using this method.
10
Hence, solving the rth level of this hierarchy using bisection can be done by semide nite programming. Indeed, for a xed r and γ given by the bisection algorithm, one simply needs to test membership of 1 X 2 d z ) fγ (z) − ( r i i
! X ·( zi2 )r i
to the set of sum of squares polynomials. This amounts to solving a semide nite program. We remark that all semide nite programming-based hierarchies available only produce an approximate solution to the optimal value of the SDP solved at level r in polynomial time. This is independent of whether they use bisection (e.g., such as the hierarchy given in Theorem 3.2 or the one based on Stengle's Positivstellensatz) or not (e.g., the Lasserre hierarchy). PnOur next theorem improves on our previous hierarchy by freeing the ( i=1 zi2 )r and taking advantage of our ability to search for an optimal
multiplier multiplier
using semide nite programming.
Theorem 3.4. Recall the de nition of Artin cones from De nition 3.1. Consider the hierarchy of optimization problems indexed by r: lr := sup
γ
γ,q (8)
n 1 X 2 d fγ (z) − ( z ) ∈ Arn,2d . r i=1 i
s.t.
Then, lr ≤ p∗ for all r, {lr } is nondecreasing, and limr→∞ lr = p∗ . Proof. Just as the previous theorem, it su ces to show that the Artin cones Arn,2d satisfy properties (a)-(d) of Theorem 2.4. The proof of property (a) follows the proof given for Theorem 3.2. Property (b) is satis ed as a (weaker) consequence of Artin's result (see Theorem 1.4). polynomial of degree
2r,
2(r + 1).
Finally, for (d),
such that
p(x) · q(x)
p(x) · q(x) isP sos for some sos q(x) · ( i x2i ) has degree Then there exists an sos form q
For (c), we have that if
P p(x) · q(x) · ( i x2i ) is r suppose that p ∈ An,2d .
then
sos, and
is sos. We have
! p(x) + (
X
x2i )d
X · q(x) = p(x) · q(x) + ( x2i )d · q(x),
i
i
which is sos as the product (resp. sum) of two sos polynomials is sos. Note that again, for any xed
r,
the level
r
of the hierarchy can be solved using
bisection which leads to a sequence of semide nite programs. Our developments in the past two sections can be phrased in terms of a Positivstellensatz.
set
Corollary 3.5
(A new Positivstellensatz).
Consider the basic semialgebraic
S := {x ∈ Rn | gi (x) ≥ 0, i = 1, . . . , m}
and a polynomial p := p(x). Suppose that S is contained within a ball of radius R. Let ηi and β be any nite upperbounds on gi (x) and, respectively, −p(x) over the set S .3 Let d be such that 2d is the smallest integer larger than or equal to the maximum 3 As
discussed at the beginning of Section 2, such bounds are very easily computable.
11
CONVERGING HIERARCHIES FOR POLYNOMIAL OPTIMIZATION
degree of p, gi , i = 1, . . . , m. Then, p(x) > 0 for all x ∈ S if and only if there exists a positive integer r such that
n X
1 x2 + h(x, s, y) − r i=1 i
m+1 X
2d r n m+1 X X s2j + y 2 · x2i + s2j + y 2
j=0
i=1
j=0
is a sum of squares, where the form h in variables (x1 , . . . , xn , s0 , . . . , sm+1 , y) is as follows: h(x, s, y) := y 2d p(x/y) + s20 y 2d−2
2
+
m X
y 2d gi (x/y) − s2i y 2d−2
2
i=1
+
(R +
m X
n m X X ηi + β)d y 2d − ( x2i + s2i )d − s2d m+1
i=1
Proof.
i=1
!2 .
i=0
This is an immediate corollary of arguments given in the proof of Theorem
2.1 and in the proof of Theorem 3.2 for the case where
γ = 0.
4. Polyá's theorem and hierarchies for POPs that are optimizationfree, LP-based, and SOCP-based. In this section, we use a result by Polyá on global positivity of even forms to obtain new hierarchies for polynomial optimization problems. In Section 4.1, we present a hierarchy that is
optimization-free, in the sense
that each level of the hierarchy only requires multiplication of two polynomials and checking if the coe cients of the resulting polynomial are nonnegative. In Section 4.2, we use the previous hierarchy to derive linear programming and second-order cone programming-based hierarchies with faster convergence. These rely on the recently developed concepts of dsos and sdsos polynomials (see De nition 4.7 and [3]), which are alternatives to sos polynomials that have been used in diverse applications to improve scalability; see [3, Section 4].
4.1. An optimization-free hierarchy of lower bounds for POPs.
The
main theorem in this section presents an optimization-free hierarchy of lower bounds for general POPs with compact feasible sets:
Theorem 4.1.
Recall the de nition of fγ (z) as given in (3), with z ∈ Rn and deg(fγ ) = 2d. Let (v, w) ∈ R2n and de ne (9)
r P oln,2d
:= {p ∈ Hn,2d |
n 1 X p(v − w ) + ( (vi4 + wi4 ))d 2r i=1 2
!
2
·(
X i
vi2 +
X
wi2 )r
2
i
has nonnegative coe cients }. Consider the hierarchy of optimization problems indexed by r: lr := sup
γ
γ (10)
n
s.t.
1 X 2 d r fγ (z) − ( z ) ∈ P oln,2d . r i=1 i
Let mr = maxi=1,...,r li . Then mr ≤ p∗ for all r, {mr } is nondecreasing, and limr→∞ mr = p∗ .
12 As before, we use bisection to obtain the optimal value lr of the hierarchy up to a xed precision
rth
level of the
(see Remark 3.3). At each step of the bisection
algorithm, one simply needs to multiply two polynomials together and check nonnegativity of the coe cients of the resulting polynomial to proceed to the next step. As a consequence, this hierarchy is optimization-free as we do not need to solve (convex) optimization problems at each step of the bisection algorithm.
To the best of our
knowledge, no other converging hierarchy of lower bounds for general POPs dispenses altogether with the need to solve convex subprograms. We also provide a Positivstellensatz counterpart to the hierarchy given above (see Corollary 4.5). This corollary implies in particular that one can always certify infeasibility of a basic semialgebraic set by recursively multiplying polynomials together and simply checking nonnegativity of the coe cients of the resulting polynomial. We now make a few remarks regarding the techniques used in the proof of Theorem 4.1. Unlike Theorems 3.2 and 3.4, we do not show that
r P oln,d
satis es properties
(a)-(d) as given in Theorem 2.4 due to some technical di culties. It turns out however that we can avoid showing properties (c) and (d) by using a result by Reznick and Powers [18] that we present below. Regarding properties (a) and (b), we show that a slightly modi ed version of (a) holds and that (b), which is the key property in Theorem 2.4, goes through as is. We note though that obtaining (b) from Polyá's result (Theorem 1.6) is not as immediate as obtaining (b) from Artin's and Reznick's results.
Indeed, unlike the theorems by Artin and Reznick (see Theorems 1.4 and
1.5) which certify global positivity of positivity of
even
any
form, Polyá's result only certi es global
forms. To make this latter result a statement about general forms,
we work in an appropriate lifted space. This is done by replacing any form variables
z ∈ Rn
by the even form
p(v 2 − w2 )
in variables
(v, w) ∈ R2n .
p(z)
in
This lifting
operation preserves nonnegativity, but unfortunately it does not preserve positivity: even if
p(z)
p(v 2 − w2 )
is pd,
always has zeros (e.g., when
v = w).
Hence, though we
now have access to an even form, we still cannot use Polyá's property as
p(v 2 − w2 )
is not positive. This is what leads us to consider the slightly more complicated form
p(v 2 − w2 ) +
1 2r (
P
Theorem 4.2
i
vi4 + wi4 )d
in (9).
Let α = (α1 , . . . , αn ) ∈ Nn , xα = , and write |α| = α1 +. . .+αn . Denote the standard simplex by ∆n . Assume that f is a form of degree 2d that is positive on ∆n and let 1 xα 1
(Powers and Reznick [18]).
. . . xnαn
λ = λ(f ) := min f (x). x∈∆n
De ne c(α) =
(2d)! α1 !...αn ! .
We have: f (x) =
X |α|=2d
aα xα =
X
bα c(α)xα .
|α|=2d
Let ||f (x)|| := max|α|=2d |bα |.4 Then, the coe cients of f (x1 , . . . , xn ) · (x1 + . . . + xn )N
are nonnegative for N > d(2d − 1) ||f (x)|| − 2d. λ 4 As
de ned,
||f ||
is a submultiplicative norm; see [23].
13
CONVERGING HIERARCHIES FOR POLYNOMIAL OPTIMIZATION
Note that here the bound is given in the case where one considers the alternative (but equivalent) formulation of Polyá's Positivstellensatz to the one given in Theorem 1.6, i.e., when one is concerned with positivity of a form over the simplex. The result can easily be adapted to the formulation where one considers global positivity of an even form as shown below.
Lemma 4.3. Let p := p(x) be an even form of degree 2d that is positive de nite. Let β > 0 be its minimum on Sx . Then, p(x1 , . . . , xn ) · (
X
x2i )N
i √
has nonnegative coe cients for N > d(2d − 1) ||p( β x)|| − 2d. Proof.
√ √ f (x1 , . . . , xn ) = p( x1 , . . . , xn ).
Since p(x) ≥ β on Sx , then f (x) ≥ β on ∆n . Indeed, by contradiction, suppose that there exists x √ Pˆ ∈ ∆n such that ˆ. Note that as ˆi = 1, we have fP(ˆ x) = β − (where > 0) and let y = x ix 2 y = 1 . Furthermore, p(y) = f (ˆ x ) = β − which contradicts the assumption. i i √ ||p( x)|| Hence, using Theorem 4.2, we have that when N > d(2d − 1) − 2d, β Let
X f (x)( xi )N i has nonnegative coe cients. Hence,
f (y 2 )(
X
yi2 )N = p(y)(
X
i
yi2 )N
i
also has nonnegative coe cients. Before we proceed with the proof of Theorem 4.1, we need the following lemma.
Lemma 4.4. (11)
Let
n n 1 X 1 X pγ,r (v, w) := fγ (v 2 − w2 ) − ( (vi2 − wi2 )2 )d + ( (vi4 + wi4 ))d , r i=1 2r i=1
where fγ is de ned as in (3) and let N (r) = d(2d − 1) ·
√ √ ||pγ,r ( v, w)|| − 2d. minSv,w pγ,r (v, w)
If fγ (z) is positive de nite, there exists rˆ such that r2 ≥ N (r), for all r ≥ rˆ. Proof. P As fγ (z) is positive de nite, there exists a positive integer r0 such fγ (z) − 1r (
2 d i zi )
is positive de nite for all
(12)
is nonnegative for all
r ≥ r0
that
and hence
1 X fγ (v 2 − w2 ) − ( (vi2 − wi2 )2 )d r i r ≥ r0 .
Recall now that
and that
||x||2 ≤
√
||x||p = (
n||x||4 .
P
i
xpi )1/p
is a norm for
p≥1
14 This implies that
X X ( vi4 + wi4 )d ≥ i
i
X X 1 ( vi2 + wi2 )2d 2d (2n) i i pγ,r ,
and hence in view of (12) and the de nition of
pγ,r (v, w) ≥
we have
X X 1 2 ( v + wi2 )2d , ∀r ≥ r0 . i 22d+1 n2d r i i
This enables us to conclude that
min pγ,r (v, w) ≥
(13)
Sv,w
1 , 22d+1 n2d r
for any
r ≥ r0 .
Further, notice that using properties of the norm, we have the following chain of inequalities for any positive integer
r:
√ √ 1 X 1 X ||pγ,r ( v, w)|| ≤ ||fγ (v − w)|| + ||( (vi − wi )2 )d || + ||( (vi2 + wi2 ))d || r 2r i i X X 2 d 2 ≤ ||fγ (v − w)|| + ||( (vi − wi ) ) || + ||( vi + wi2 )d || =: cγ . i
i
As a consequence, combining this with the de nition of
N (r)
and (13), we have
N (r) ≤ d(2d − 1)22d+1 rn2d cγ , ∀r ≥ r0 . Now taking
rˆ = max(r0 , dd(2d − 1)22d+1 n2d cγ e),
we have
r2 ≥ N (r), ∀r ≥ rˆ.
We now proceed with the proof of Theorem 4.1.
Proof of Theorem 4.1.
{mr } is γ is such
By de nition, the sequence
show that it is upperbounded by
p∗
by showing that if
nondecreasing.
We
that
1 X 2 d r fγ (z) − ( z ) ∈ P oln,2d , r i i for some
r,
then
fγ
must be positive de nite. Then Theorem 2.1 gives us that
a strict lower bound on (1). As
mr ≤ p∗ , ∀r. Assume that γ
p∗ > γ
for any such
γ,
we have that lr
≤ p∗ , ∀r
γ
is
and
hence
is such that
1 X 2 d r fγ (z) − ( z ) ∈ P oln,2d r i i for some
r.
By de nition of
r P oln,2d
and as
(
P
i
vi2 +
P
i
wi2 )r
2
is nonnegative, we get
that the form
1 X 1 X 4 fγ (v 2 − w2 ) − ( (vi2 − wi2 )2 )d + ( v + wi4 )d r i 2r i i is nonnegative. This implies that (14)
1 X 1 fγ (v 2 − w2 ) − ( (vi2 − wi2 )2 )d ≥ − r i 2r
for
(v, w) ∈ {(v, w) |
X i
vi4 +
X i
wi4 = 1},
15
CONVERGING HIERARCHIES FOR POLYNOMIAL OPTIMIZATION
which gives
1 1 X 2 d z ) ≥ − , ∀z ∈ Sz . fγ (z) − ( r i i 2r
(15)
zˆ ∈ Sz such that (15) does not hold. Then, let − + − zˆ+ = max(ˆ z , 0) and z ˆ = max(−ˆ z , √ √ 0). Note that both zˆ and zˆ are nonnegative so + − we can take v ˆ = zˆ . We further have that as zˆ ∈ Sz and zˆ = vˆ2 − w ˆ2 , P P ˆ 4= zˆ and w 4 2 2 ˆi + i w ˆi = 1. Substituting zˆ by vˆ − w ˆ in (15) then violates (14). Using (15), iv
Indeed, suppose that there exists
we conclude that
1 , ∀z ∈ Sz 2r
fγ (z) ≥ and that
fγ
is positive de nite.
limr→∞ mr = p∗ .
We now show that the hierarchy converges, i.e., that
γ
this, we show that if 2.1, if
fγ (z)
is positive de nite, then there exists
fγ (z) − fγ is pd, any r ≥ r0 .
To do
is a strict lower bound on (1), or equivalently from Theorem
r0
such that
1 X 2 d r0 ( z ) ∈ P oln,2d . r0 i i
Pn 1 2 d r0 such that fγ (z) − ( i=1 zi ) is pd r P 1 fγ (v 2 − w2 ) − r ( i (vi2 − wi2 )2 )d is nonnegative and
Since
there exists a positive integer
for
This implies that
1 X 1 X fγ (v 2 − w2 ) − ( (vi2 − wi2 )2 )d + ( (vi4 + wi4 ))d r i 2r i is positive de nite for 4.4, for any
r ≥ r0 ,
r ≥ r0 .
Using Lemma 4.3 and the de nition of
1 X 1 X fγ (v − w ) − ( (vi2 − wi2 )2 )d + ( (vi4 + wi4 ))d r i 2r i 2
·(
r0 = max{r0 , rˆ}
and considering
pγ,r0
X
vi2 +
X
i
has nonnegative coe cients. From Lemma 4.4, there exists Taking
in Lemma
!
2
r2 ≥ N (r).
N (r)
we have that
rˆ such
wi2 )dN (r)e
i that
r ≥ rˆ implies
as de ned in (11), we get
that
pγ,r0 (v, w)(
X
vi2 +
X
i
= pγ,r0 (v, w)(
wi2 )r
02
i
X
vi2
+
i
X
X X 0 02 0 wi2 )dN (r )e · ( vi2 + wi2 )r −dN (r )e
i
i
i
has nonnegative coe cients, which is the desired result. This is because
pγ,r0 (v, w)(
X
vi2 +
i
X
0
wi2 )dN (r )e
i
r0 ≥ r0 , and X X 02 0 ( vi2 + wi2 )r −dN (r )e
has nonnegative coe cients as
i has nonnegative coe cients as
r0 ≥ rˆ,
i and that the product of two polynomials with
nonnegative coe cients has nonnegative coe cients.
16
Corollary 4.5
semialgebraic set
(An optimization-free Positivstellensatz).
Consider the basic
S := {x ∈ Rn | gi (x) ≥ 0, i = 1, . . . , m}
and a polynomial p := p(x). Suppose that S is contained within a ball of radius R. Let ηi and β be any nite upperbounds on gi (x) and, respectively, −p(x) over the set S .5 Let d be such that 2d is the smallest even integer larger than or equal to the maximum degree of p, gi , i = 1, . . . , m. Then, p(x) > 0 for all x ∈ S if and only if there exists a positive integer r such that ! n+m+3 n+m+3 1 X 1 X 2 2 2 d 4 4 d (vi − wi ) ) + ( (vi + wi )) h(v − w ) − ( r i=1 2r i=1 !r2 n+m+3 n+m+3 X X 2 2 · vi + wi 2
2
i=1
i=1
has nonnegative coe cients, where the form h := h(z) in variables (z1 , . . . , zn+m+3 ) := (x1 , . . . , xn , s0 , . . . , sm+1 , y)
is as follows: h(x, s, y) := y 2d p(x/y) + s20 y 2d−2
2
+
m X
y 2d gi (x/y) − s2i y 2d−2
2
i=1
+
(R +
m X
ηi + β) y
i=1
Proof.
d 2d
n m X X −( x2i + s2i )d − s2d m+1 i=1
!2 .
i=0
This is an immediate corollary of arguments given in the proof of Theo-
rem 2.1 and in the proof of Theorem 4.1 for the case where
γ = 0.
4.2. Linear programming and second-order cone programming-based hierarchies for POPs. In this section, we present a linear programming and a second-order cone programming-based hierarchy for general POPs which by construction converge faster than the hierarchy presented in Section 4.1. These hierarchies are based on the recently-introduced concepts of dsos and sdsos polynomials [3] which we brie y revisit below to keep the presentation self-contained.
Definition 4.6. • •
A symmetric matrix M P is said to be if Mii ≥ j6=i |Mij | for all i. scaled diagonally dominant (sdd) if there exists a diagonal matrix D, with positive diagonal entries, such that DAD is dd. diagonally dominant (dd)
We have the following implications as a consequence of Gershgorin's circle theorem: (16)
M
Requiring
M
to be dd (resp.
dd ⇒ M sdd ⇒ M 0. sdd) can be encoded via a linear program (resp.
a
second-order cone program) (see [3] for more details). These notions give rise to the concepts of dsos and sdsos polynomials.
5 Once
again, as discussed at the beginning of Section 2, such bounds are very easily computable.
17
CONVERGING HIERARCHIES FOR POLYNOMIAL OPTIMIZATION
Definition 4.7
Let z(x) = (xd1 , xd−1 x2 , . . . , xdn )T be the vector of mono1 mials in (x1 , . . . , xn ) of degree d. A form p ∈ Hn,2d is said to be • diagonally-dominant-sum-of-squares (dsos) if it admits a representation ([3]).
p(x) = z T (x)Qz(x), where Q is a dd matrix. •
scaled-diagonally-dominant-sum-of-squares (sdsos)
tion
if it admits a representa-
p(x) = z T (x)Qz(x), where Q is a sdd matrix.
The following implications are a consequence of (16):
p(x)
(17)
dsos
⇒ p(x)
sdsos
⇒ p(x)
sos
⇒ p(x)
nonnegative.
Given the fact that our Gram matrices and polynomials are related to each other via linear equalities, it should be clear that optimizing over the set of dsos (resp. sdsos) polynomials is an LP (resp. SOCP). We now present our LP and SOCP-based hierarchies for POPs.
Corollary 4.8.
Recall the de nition of fγ (z) as given in (3), with z ∈ Rn and deg(f ) = 2d, and let pγ,r be as in (11). Consider the hierarchy of optimization problems indexed by r: lr := sup
γ
γ,q
s.t.
(18)
pγ,r (v, w) · q(v, w) is s/dsos q(v, w) is s/dsos and of degree 2r2 .
Let mr = maxi=1,...,r li . Then, mr ≤ p∗ for all r, {mr } is nondecreasing, and we have limr→∞ mr = p∗ . Proof.
p ∈ Hn,2d p(x) = z(x)T Qz(x) where Q is diagonal As such a Q is dd (and also sdd), we conclude
This is an immediate consequence of the fact that any even form
with nonnegative coe cients can be written as and has nonnegative (diagonal) entries. that
p
is dsos (and also sdsos). The corollary then follows from Theorem 4.1.
Note that similarly to our previous hierarchies, one must proceed by bisection on
γ
to solve the level
r
of the hierarchy. At each step of the hierarchy, we solve a linear
program (resp. second-order cone program) that searches for the coe cients of make
q
dsos (resp. sdsos) and
pγ,r · q
q
that
dsos (resp. sdsos).
There is a trade-o between the hierarchies developed in this subsection and the one developed in the previous subsection: the hierarchy of Section 4.1 is optimizationfree whereas those of Section 4.2 use linear or second-order cone programming. Hence the former hierarchy is faster to run at each step. However, the latter hierarchies could potentially take fewer levels to converge.
This is similar to the trade-o observed
between the hierarchies presented in Theorems 3.2 and 3.4.
5. Open problems.
To conclude, we present two open problems spawned by
the writing of this paper. The rst one concerns the assumptions needed to construct our hierarchies.
Open problem 1.
Theorems 2.1 and 2.4 require that the feasible set
given in (1) be contained in a ball of radius
R.
S
of the POP
Can these theorems be extended to
the case where there is no compactness assumption on
S?
The second open problem is linked to the Artin and Reznick cones presented in De nition 3.1.
18
Open problem 2.
r Rn,2d are convex for all r . r We are unable to prove however that Artin cones An,2d are convex (even though they satisfy properties (a)-(d) of Theorem 2.4 like Reznick cones do). Are Artin cones convex for all
r?
As mentioned before, Reznick cones
We know that they are convex for
r=0
and for
r
large enough as
they give respectively the sos and psd cones (see [12] for the latter claim). However, we do not know the answer already for
Acknowledgments.
r = 1.
We are grateful to Pablo Parrilo for very insightful com-
ments, particularly as regards Section 4 and the observation that any form can be made even by only doubling the number of variables and the degree.
REFERENCES
On the di culty of deciding asymptotic stability of cubic homogeneous vector elds, in Proceedings of the American Control Conference, 2012. A. Ahmadi and A. Majumdar, Some applications of polynomial optimization in operations research and real-time decision making, Optimization Letters, 10 (2016), pp. 709729. A. Ahmadi and A. Majumdar, DSOS and SDSOS optimization: more tractable alternatives to sum of squares and semide nite optimization, (2017). Available on arXiv at
[1] A. A. Ahmadi, [2] A. [3] A.
https://arxiv.org/abs/1706.02586. [4] E. Artin,
Über die Zerlegung de niter Funktionen in Quadrate,
in Abhandlungen aus dem
mathematischen Seminar der Universität Hamburg, vol. 5, Springer, 1927, pp. 100115.
Constructive proofs of some Positivstellensätze for compact semialgebraic subsets of Rd , Journal of Optimization Theory and Applications, 158 (2013), pp. 410418. G. Blekherman, P. A. Parrilo, and R. R. Thomas, Semide nite Optimization and Convex Algebraic Geometry, SIAM, 2012. E. de Klerk, M. Laurent, and P. Parrilo, On the equivalence of algebraic approaches to the minimization of forms on the simplex, Positive Polynomials in Control, (2005),
[5] G. Averkov, [6] [7]
pp. 580580.
Über die Zerlegung strikte de niter Formen in Quadrate, Commentarii Mathe-
[8] W. Habicht,
matici Helvetici, 12 (1939), pp. 317322. [9] J. B. Lasserre,
Global optimization with polynomials and the problem of moments,
SIAM
Journal on Optimization, 11 (2001), pp. 796817. [10] J. B. Lasserre,
Moments, Positive Polynomials and their Applications, vol. 1, World Scien-
ti c, 2009. [11] M. Laurent,
Sums of squares, moment matrices and optimization over polynomials, in Emerg-
ing applications of algebraic geometry, Springer, 2009, pp. 157270.
An elementary recursive bound for eective Positivstellensatz and Hilbert 17-th problem, arXiv preprint arXiv:1404.2338, (2014). Nie and M. Schweighofer, On the complexity of Putinar's Positivstellensatz, Journal of
[12] H. Lombardi, D. Perrucci, and M.-F. Roy, [13] J.
Complexity, 23 (2007), pp. 135150.
Quadratic programming with one negative eigenvalue is NP-hard, Journal of Global Optimization, 1 (1991), pp. 1522. P. A. Parrilo, Structured semide nite programs and semialgebraic geometry methods in robustness and optimization, PhD thesis, California Institute of Technology, May 2000. P. A. Parrilo, Semide nite programming relaxations for semialgebraic problems, Mathemat-
[14] P. M. Pardalos and S. A. Vavasis, [15] [16]
ical Programming, 96 (2003), pp. 293320. [17] G. Pólya,
Über positive Darstellung von Polynomen,
Vierteljschr. Naturforsch. Ges. Zürich,
73 (1928), pp. 141145.
A new bound for Pólya's theorem with applications to polynomials positive on polyhedra, Journal of pure and applied algebra, 164 (2001), pp. 221229. Putinar, Positive polynomials on compact semi-algebraic sets, Indiana University Math-
[18] V. Powers and B. Reznick, [19] M.
ematics Journal, 42 (1993), pp. 969984. [20] B. Reznick,
Uniform denominators in Hilbert's 17th problem, Math Z., 220 (1995), pp. 7597. The k-moment problem for compact semi-algebraic sets, Mathematische An-
[21] K. Schmüdgen,
nalen, 289 (1991), pp. 203206. [22] M. Schweighofer,
An algorithmic approach to Schmüdgen's Positivstellensatz,
Pure and Applied Algebra, 166 (2002), pp. 307319. [23] M. Schweighofer,
Journal of
On the complexity of Schmüdgen's Positivstellensatz, Journal of Complex-
ity, 20 (2004), pp. 529543.
CONVERGING HIERARCHIES FOR POLYNOMIAL OPTIMIZATION
[24] M. Schweighofer,
Optimization of polynomials on compact semialgebraic sets, SIAM Journal
on Optimization, 15 (2005), pp. 805825. [25] G. Stengle,
19
A Nullstellensatz and a Positivstellensatz in semialgebraic geometry, Mathema-
tische Annalen, 207 (1974), pp. 8797.
