Commit df9af168 by nanored

### ex 4 dm 2

parent cffb4be8
 ... ... @@ -8,6 +8,7 @@ \usepackage{kpfonts} \usepackage{tikz} \usepackage{bbm} \usepackage{hyperref} \title{ \noindent\rule{\linewidth}{0.4pt} ... ... @@ -19,6 +20,9 @@ \newcommand{\trans}{\mathsf{T}} \newcommand{\syst}[2]{\left\{ \begin{array}{#1} #2 \end{array} \right.} \newcommand{\diag}{\text{\textbf{diag}}~} \newcommand{\pmat}[1]{\begin{pmatrix} #1 \end{pmatrix}} \newcommand{\R}{\mathbb{R}} \begin{document} ... ... @@ -153,7 +157,10 @@ Where $h$ is a strictly concave function. We compute the gradient of $h$ with respect to $y$ : $$\nabla_y h(\nu, y) = \nu - 2 y$$ Then $h$ is maximal for $y = \frac{1}{2} \nu$ and we obtain : $$\| \nu \|_2^{2 \, *} = \frac{1}{4} \| \nu \|_2^2$$ \label{conj2} \| \nu \|_2^{2 \, *} = \frac{1}{4} \| \nu \|_2^2 Then our dual can be written : \begin{center} \fbox{$\displaystyle \text{(Dual RLS)} \quad \begin{array}{llr} ... ... @@ -174,9 +181,9 @@ In this case the optimal value is : $$p_2^* = \dfrac{1}{n \tau} \sum_{i = 1}^n \mathcal{L}(\omega^*, x_i, y_i) + \dfrac{1}{2} \| \omega^* \|_2^2 = \dfrac{1}{\tau} p_1(\omega^*)$$ Where$p_1(\omega)$is the value of the point$\omega$in the problem (Sep. 1). \\ Finally we can remark that for every$\omega$, the point$\left( \omega, \left( \mathcal{L}(\omega, x_i, y_i) \right)_{i = 1}^n \right)$is feasible for the problem (Sep. 2). We showed before that the optimal value can be expressed in this form then : Finally we can remark that for every$\omega$, the point$\left( \omega, \left( \mathcal{L}(\omega, x_i, y_i) \right)_{i = 1}^n \right)$is feasible for the problem (Sep. 2). We showed before that the optimal value can be expressed in this form then: $$p_2^* = \inf_\omega \dfrac{1}{\tau} p_1(\omega) = \dfrac{1}{\tau} p_1^* = \dfrac{1}{\tau} p_1(\omega^*)$$ We can now conclude : We can now conclude: \begin{center} \fbox{\parbox{0.93\linewidth}{\centering The problem (Sep. 2) as a solution$(\omega^*, z^*)$of value$p_2^*$such that$\omega^*$is optimal for (Sep. 1),$z^*$is the vector of losses and$p_1^* = \tau p_2^*$. ... ... @@ -184,6 +191,73 @@ \end{center} \paragraph{2.} We first compute the dual function : $$\begin{array}{lll} g(\omega, z, \lambda, \pi) & = & \inf_{\omega, z} \dfrac{1}{n \tau} \mathbf{1}^\trans z + \dfrac{1}{2} \| \omega \|_2^2 + \mathbf{1}^\trans \lambda - \lambda^\trans \diag(y) X \omega - \lambda^\trans z - \pi^\trans z \\ & = & \mathbf{1}^\trans \lambda - \dfrac{1}{2} \left\| 2 X^\trans \diag(y) \lambda \right\|_2^{2 \, *} + \inf_z \left( \dfrac{1}{n \tau} \mathbf{1} - \lambda - \pi \right)^\trans z \end{array}$$ Where$X \in \R^{n \times d}$is the matrix with$x_i^\trans$as$i$-th row and$ \| . \|_2^{2 \, *}$is the conjugate of the 2-norm as seen in the previous exercise (eq \eqref{conj2}): $$X = \pmat{x_1^\trans \\ \vdots \\ x_n^\trans } \qquad \| v \|_2^{2 \, *} = \dfrac{1}{4} \| v \|_2^2$$ We then obtain: $$g(\omega, z, \lambda, \pi) = \syst{ll}{ \mathbf{1}^\trans \lambda - \dfrac{1}{2} \left\| X^\trans \diag(y) \lambda \right\|_2^2 & \text{If } \; \lambda = \frac{1}{n \tau} \mathbf{1} - \pi \\[2mm] - \infty & \text{Otherwise} }$$ This gives the following dual problem: $$\begin{array}{llr} \max_{\lambda, \pi} & \mathbf{1}^\trans \lambda - \dfrac{1}{2} \left\| X^\trans \diag(y) \lambda \right\|_2^2 \\[2mm] \text{s.t.} & \lambda = \frac{1}{n \tau} \mathbf{1} - \pi \\[1mm] & \lambda \succeq 0, \; \pi \succeq 0 \end{array}$$ We can inject the inequality on$\pi$in the first equality and remove this variable from the dual: \begin{center} \fbox{$ \text{(Dual Sep. 2)} \quad \begin{array}{llr} \max_{\lambda} & \mathbf{1}^\trans \lambda - \dfrac{1}{2} \left\| X^\trans \diag(y) \lambda \right\|_2^2 \\[2mm] \text{s.t.} & 0 \preceq \lambda \preceq \frac{1}{n \tau} \mathbf{1} \end{array} $} \end{center} \section*{Exercise 4 (Robust linear programming)} \paragraph{} We denote by$p^*(x)$the value: $$p^*(x) = \sup_{a \in \mathcal{P}} a^\trans x = - \inf_{a \in \mathcal{P}} - a^\trans x$$ Then$p(x)$is the opposite of the optimal value of a linear program. We know that strong duality holds in this case and we compute the dual function associated to (P)$\inf_{a \in \mathcal{P}} - a^\trans x$: $$g(a, z) = \inf_a -a^\trans x + z^\trans C^\trans a - z^\trans d = \syst{ll}{ -z^\trans d & \text{If } \, Cz = x \\ - \infty & \text{Otherwise} }$$ This gives the following dual of (P): $$(D) \quad \begin{array}{llr} \max_z & - d^\trans z \\ \text{s.t.} & Cz = x \\ & z \succeq 0 \end{array}$$ The optimal value of this dual is$d^*(x) = -p^*(x)$by strong duality. \paragraph{} Because the objective function of the two LP we want to show the equivalence are the same, it is sufficient to show that : \label{equiv} p^*(x) \leqslant b \; \Leftrightarrow \; \exists z \succeq 0, \, C z = x \text{ and } d^\trans z \leqslant b \textit{Remark that I wrote$Cz =x$instead of$C^\trans z = x$as suggested in the exercise statement, which is certainly wrong} \\ Now we suppose that$x$satisfies$p^*(x) \leqslant b$. We know that there exists$z^* \succeq 0$such that: $$C z^* = x \qquad \text{And} \qquad - d^\trans z^* = d^*(x) = - p^*(x) \geqslant -b \; \Leftrightarrow \; d^\trans z^* \leqslant b$$ Inversely, now we suppose that$p^*(x) > b$. Let$z \succeq 0$such that$Cz = x$. Thus,$z$is in the polytope of (D). This gives : $$- d^\trans z \leqslant d^*(x) = -p^*(x) < -b \; \Leftrightarrow \; d^\trans z > b$$ Then there is no$z \succeq 0$such that$C z = x \text{ and } d^\trans z \leqslant b$. \\ We just proved the equivalence \eqref{equiv} that allows us to conclude : \begin{center} \fbox{$ \displaystyle \begin{array}{llr} \max_x & c^\trans x \\ \text{s.t.} & \sup_{a \in \mathcal{P}} a^\trans x \leqslant b \end{array} \qquad \Leftrightarrow \qquad \begin{array}{llr} \min_{x, z} & c^\trans x \\ \text{s.t.} & d^\trans z \leqslant b \\ & Cz = x \\ & z \succeq 0 \end{array} \$} \end{center} \end{document} \ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!