Commit fd45be23 authored by nanored's avatar nanored

end of hw 2

parent df9af168
......@@ -145,11 +145,11 @@
\end{array} $$
We then compute the dual function :
$$ \begin{array}{lll}
g(\nu) & = & \inf_{x, y} \| y \|_2^2 - \nu^\trans y + \nu^\trans A x + \| x \|_1 - \nu^\trans b \\[2mm]
& = & \| \nu \|_2^{2 \, *} + \left\| - A^\trans \nu \right\|_1^* - \nu^\trans b \\[2mm]
g(\nu) & = & \inf_{x, y} \| y \|_2^2 - \nu^\trans y + \nu^\trans A x + \| x \|_1 + \nu^\trans b \\[2mm]
& = & - \| \nu \|_2^{2 \, *} - \left\| - A^\trans \nu \right\|_1^* + \nu^\trans b \\[2mm]
& = & \syst{ll}{
\| \nu \|_2^{2 \, *} - \nu^\trans b & \text{If } \, \left\| A^\trans \nu \right\|_{\infty} \leqslant 1 \\
+ \infty & \text{Otherwise}
\nu^\trans b - \| \nu \|_2^{2 \, *} & \text{If } \, \left\| A^\trans \nu \right\|_{\infty} \leqslant 1 \\
- \infty & \text{Otherwise}
}
\end{array} $$
Now we compute the conjugate of the square of the 2-norm :
......@@ -164,7 +164,7 @@
Then our dual can be written :
\begin{center}
\fbox{$ \displaystyle \text{(Dual RLS)} \quad \begin{array}{llr}
\max_\nu & \frac{1}{4}\| \nu \|_2^2 - b^\trans \nu \\[2mm]
\max_\nu & b^\trans \nu - \frac{1}{4}\| \nu \|_2^2 \\[2mm]
\text{s.t.} & \left\| A^\trans \nu \right\|_{\infty} \leqslant 1
\end{array} $}
\end{center}
......@@ -175,7 +175,7 @@
First, we can remark that (Sep. 2) is feasible by simply taking the point $(0, \mathbf{1})$ and bounded because the objective function is non-negative for $z \succeq 0$. \\
Let $(\omega, z)$ a feasible point. Suppose that there exists $i$ such that $z_i > \mathcal{L}(\omega, x_i, y_i)$. Then we let $z'$ such that $z'_j = z_j$ for $j \neq i$ and $z'_i = \mathcal{L}(\omega, x_i, y_i)$. We have in particular : \vspace{-2mm}
$$ z'_i \geqslant 0 \qquad z'_i \geqslant 1 - y_i (\omega^\trans x_i) $$
And the other constraints are also verified because $z$ is feasible. Because the factor of $z_i$ is strictly positive in the objective function and because $z'_i < z_i$, the value associated to the point $(\omega, z')$ is strictly inferior to the value of $(\omega, z)$. \\
Furthermore the other constraints are also verified because $z$ is feasible. Because the factor of $z_i$ is strictly positive in the objective function and because $z'_i < z_i$, the value associated to the point $(\omega, z')$ is strictly inferior to the value of $(\omega, z)$. \\
Then for $(\omega^*, z^*)$ optimal we have $z_i^* \leqslant \mathcal{L}(\omega^*, x_i, y_i)$ for every $i$. Furthermore, because of constraints, this inequality is in fact an equality. So : \vspace{-2mm}
$$ z_i^* = \mathcal{L}(\omega^*, x_i, y_i) $$
In this case the optimal value is :
......@@ -222,7 +222,7 @@
\paragraph{}
We denote by $p^*(x)$ the value:
$$ p^*(x) = \sup_{a \in \mathcal{P}} a^\trans x = - \inf_{a \in \mathcal{P}} - a^\trans x $$
Then $p(x)$ is the opposite of the optimal value of a linear program. We know that strong duality holds in this case and we compute the dual function associated to (P) $\inf_{a \in \mathcal{P}} - a^\trans x$:
Then $p^*(x)$ is the opposite of the optimal value of a linear program. We know that strong duality holds in this case and we compute the dual function associated to (P) $\inf_{a \in \mathcal{P}} - a^\trans x$:
$$ g(a, z) = \inf_a -a^\trans x + z^\trans C^\trans a - z^\trans d = \syst{ll}{
-z^\trans d & \text{If } \, Cz = x \\
- \infty & \text{Otherwise}
......@@ -250,7 +250,7 @@
We just proved the equivalence \eqref{equiv} that allows us to conclude :
\begin{center}
\fbox{$ \displaystyle \begin{array}{llr}
\max_x & c^\trans x \\
\min_x & c^\trans x \\
\text{s.t.} & \sup_{a \in \mathcal{P}} a^\trans x \leqslant b
\end{array} \qquad \Leftrightarrow \qquad \begin{array}{llr}
\min_{x, z} & c^\trans x \\
......@@ -259,5 +259,69 @@
& z \succeq 0
\end{array} $}
\end{center}
\subsection*{Exercise 5 (Boolean LP)}
\paragraph{1.}
We first compute the Lagrangian of our problem: \vspace{-3mm}
$$ L(x, \lambda, \nu) = c^\trans x + \lambda^\trans (Ax - b) + \sum_{i = 1}^n \nu_i x_i (1 - x_i) \vspace{-2mm} $$
Now we compute the gradient with respect to $x_i$ of the Lagrangian:
$$ \nabla_{x_i} L(x, \lambda, \nu) = c_i + a_i^\trans \lambda + \nu_i (1 - 2 x_i) \qquad \text{where } \; A = \pmat{a_1 & \cdots & a_n} $$
This gradient depends only on $x_i$ then the minimum of $L$ w.r.t $x$ is obtained by optimizing on each $x_i$ independently. We can remark that if $\nu_i > 0$, then $L(\alpha e_i, \lambda, \nu)$ tends to $- \infty$ when $\alpha$ tends to $+\infty$. So, for $L$ to be bounded, we need $\nu \preceq 0$. In this case $L$ is convex in $x$ and we just need to find the vanishing point of the gradient to obtain the minimum of $L$:
$$ \nabla_{x_i} L(x, \lambda, \nu) = 0 \quad \Leftrightarrow \quad \left( \nu_i < 0 \, \text{ and } \, x_i = \dfrac{c_i + a_i^\trans \lambda + \nu_i}{2 \nu_i} \right) \; \text{or} \; \left( \nu_i = 0 \, \text{ and } \, c_i + a_i^\trans \lambda = 0 \right) $$
Now we write:
$$ L(x, \lambda, \nu) = - \lambda^\trans b + \sum_{i = 1}^n L_i(x_i, \lambda, \nu_i) \qquad \text{where } \; L_i(x_i, \lambda, \nu_i) = \left( c_i + a_i^\trans \lambda + \nu_i - \nu_i x_i \right) x_i $$
Such that the dual function wan be written:
$$ g(\lambda, \nu) = \inf_x L(x, \lambda, \nu) = - \lambda^\trans b + \sum_{i = 1}^n \inf_{x_i} L_i(x_i, \lambda, \nu_i) = -\lambda^\trans b + \sum_{i = 1}^n g_i(\lambda, \nu_i) $$
With the following expression of $g_i$, from what we have done before:
$$ g_i(\lambda, \nu_i) = \syst{ll}{
L_i \left( \dfrac{c_i + a_i^\trans \lambda + \nu_i}{2 \nu_i}, \, \lambda, \, \nu_i \right) & \text{If } \; \nu_i < 0 \\
0 & \text{If } \; \nu_i = 0 \; \text{ and } \; c_i + a_i^\trans \lambda = 0 \\
- \infty & \text{Otherwise}
} $$
We simplify the case $\nu_i < 0$: \vspace{-3mm}
$$ \nu_i < 0 \: \Rightarrow \; g_i(\lambda, \nu_i) = \dfrac{1}{4 \nu_i} \left( c_i + a_i^\trans \lambda + \nu_i \right)^2 $$
Because $g$ depends on $\nu_i$ only in the term $g_i(\lambda, \nu_i)$ we can simplify the expression of the Lagrange dual by maximizing each $g_i$ with respect to $\nu_i$ and removing each $\nu_i$ from the dual variables. \\
From the hint given in the question we have:
$$ \sup_{\nu_i < 0} g_i(\lambda, \nu_i) = \min \left\{ 0, \, c_i + a_i^\trans \lambda \right\} $$
Furthermore :
$$ c_i + a_i^\trans \lambda = 0 \quad \Rightarrow \quad g_i(\lambda, 0) = 0 = \min \left\{ 0, \, c_i + a_i^\trans \lambda \right\} $$
Now we can remove the variable $\nu_i$ to obtain:
$$ g_i(\lambda) = \sup_{\nu_i} g_i(\lambda, \nu_i) = \min \left\{ 0, \, c_i + a_i^\trans \lambda \right\} $$
Finally we obtain the following expression for the Lagrange dual:
\begin{center}
\fbox{$ \displaystyle g(\lambda) = -b^\trans \lambda + \sum_{i = 1}^n \min \left\{ 0, \, c_i + a_i^\trans \lambda \right\} $}
\end{center}
\paragraph{2.}
We compute the Lagrange dual of the LP relaxation:
$$ h(x, \alpha, \beta, \gamma) = \inf_x \; c^\trans x + \alpha^\trans Ax - \alpha^\trans b - \beta^\trans x + \gamma^\trans x - \gamma^\trans \mathbf{1} = \syst{ll}{
- \alpha^\trans b - \gamma^\trans \mathbf{1} & \text{If } \; c + A^\trans \alpha + \gamma = \beta \\
- \infty & \text{Otherwise}
} $$
This gives the follwing dual problem:
$$ \begin{array}{llr}
\max_{\alpha, \beta, \gamma} & - \alpha^\trans b - \gamma^\trans \mathbf{1} \\
\text{s.t.} & c + A^\trans \alpha + \gamma = \beta \\
& \alpha, \beta, \gamma \succeq 0
\end{array} $$
We can inject the non-negativity of $\beta$ in the equality constraint and remove this variable:
$$ \begin{array}{llr}
\max_{\alpha, \gamma} & - \alpha^\trans b - \gamma^\trans \mathbf{1} \\
\text{s.t.} & c + A^\trans \alpha + \gamma \succeq 0 \\
& \alpha, \gamma \succeq 0
\end{array} $$
We also remark that the objective function is strictly decreasing with respect to $\gamma$. Then the optimal is obtain for $\gamma$ as small as possible when $\alpha$ is fixed. This leads us to:
$$ \gamma_i = \max \left\{ 0, \, - c_i - a_i^\trans \alpha \right\} = - \min \left\{ 0, \, c_i + a_i^\trans \alpha \right\} $$
Now we are able to simplify again our dual:
$$ \text{(D2)} \quad \begin{array}{llr}
\max_{\alpha} & g(\alpha) = - \alpha^\trans b + \sum_{i = 1}^n \min \left\{ 0, \, c_i + a_i^\trans \alpha \right\} \\
\text{s.t.} & \alpha \succeq 0
\end{array} $$
We recognize the dual obtain in the previous question for the Lagrange relaxation.
\begin{center}
\fbox{The lower bound obtained via Lagrangian relaxation, and via the LP relaxation (2), are
the same.}
\end{center}
\end{document}
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment