[原创]支持向量机（SVM）的推导及求解
（一）推导
SVM的原问题为：对于给定的线性可分的训练样本集：
![\[{\rm{S}} = \{ ({x_1},{y_1}),({x_2},{y_2}),...,({x_l},{y_l})\} \]](/user_files/duweifu/epics/f69d9291149e144702574582deda23e9ca98408e.png "\[{\rm{S}} = \{ ({x_1},{y_1}),({x_2},{y_2}),...,({x_l},{y_l})\} \]")
，
我们的目标是寻找适当的的参数![\[({w^*},{b^*})\]](/user_files/duweifu/epics/f46450b4e1441ca385212fb809ce2bed725a090d.png "\[({w^*},{b^*})\]")
，使得
，使得![\[\min imis{e_{w,b}}\quad \left\langle {w \cdot w} \right\rangle \]](/user_files/duweifu/epics/2b4aafad7c4d13d3bbd06e606a4b7088170d97cb.png "\[\min imis{e_{w,b}}\quad \left\langle {w \cdot w} \right\rangle \]")
![\[subject\to\quad {y_i}\left( {\left\langle {w \cdot {x_i}} \right\rangle + b} \right) \ge 1\quad i = 1,2,...,l\]](/user_files/duweifu/epics/1e8d7cc3d2a3b3c942b979de74f244f5d8926dad.png "\[subject\to\quad {y_i}\left( {\left\langle {w \cdot {x_i}} \right\rangle + b} \right) \ge 1\quad i = 1,2,...,l\]")
相应地，其拉格朗日函数为：
![\[L(w,b,\alpha ) = \frac{1}{2}\left\langle {w \cdot w} \right\rangle - \sum\limits_{i = 1}^l {{\alpha _i}[{y_i}\left( {\left\langle {w \cdot {x_i}} \right\rangle + b} \right) - 1]} \]](/user_files/duweifu/epics/3722b983901547170576bc083c9ca635d532f0d5.png "\[L(w,b,\alpha ) = \frac{1}{2}\left\langle {w \cdot w} \right\rangle - \sum\limits_{i = 1}^l {{\alpha _i}[{y_i}\left( {\left\langle {w \cdot {x_i}} \right\rangle + b} \right) - 1]} \]")
分别对![\[w\]](/user_files/duweifu/epics/d92a542f3010981badf6c016a5955d5b238c2dab.png "\[w\]")
和![\[b\]](/user_files/duweifu/epics/863bc2abfcdf54332b3f1d66ee51bf6049b5c8aa.png "\[b\]")
求偏导：
和求偏导：![\[\frac{{\partial L(w,b,a)}}{{\partial w}} = w - \sum\limits_{i = 1}^l {{y_i}{\alpha _i}{x_i}} = 0\]](/user_files/duweifu/epics/c7f2d6c7104630fce1ca11031d4b3bbd43593491.png "\[\frac{{\partial L(w,b,a)}}{{\partial w}} = w - \sum\limits_{i = 1}^l {{y_i}{\alpha _i}{x_i}} = 0\]")
![\[\frac{{\partial L(w,b,a)}}{{\partial b}} = \sum\limits_{i = 1}^l {{y_i}{\alpha _i}} = 0\]](/user_files/duweifu/epics/ee426e8a8d794f855927890752da01dda259039e.png "\[\frac{{\partial L(w,b,a)}}{{\partial b}} = \sum\limits_{i = 1}^l {{y_i}{\alpha _i}} = 0\]")
带回原拉格朗日函数：
![\[\begin{array}{l} L(w,b,\alpha ) = \frac{1}{2}\left\langle {w\cdotw} \right\rangle - \sum\limits_{i = 1}^l {{\alpha _i}[{y_i}\left( {\left\langle {w\cdot{x_i}} \right\rangle + b} \right) - 1]} \\ \quad \quad \quad \quad = \frac{1}{2}\sum\limits_{i,j = 1}^l {{y_i}{y_j}{\alpha _i}{\alpha _j}\left\langle {{x_i},{x_j}} \right\rangle - } \sum\limits_{i,j = 1}^l {{y_i}{y_j}{\alpha _i}{\alpha _j}\left\langle {{x_i},{x_j}} \right\rangle } + \sum\limits_{i = 1}^l {{\alpha _i}} \\ \quad \quad \quad \quad = \sum\limits_{i = 1}^l {{\alpha _i}} - \frac{1}{2}\sum\limits_{i,j = 1}^l {{y_i}{y_j}{\alpha _i}{\alpha _j}\left\langle {{x_i},{x_j}} \right\rangle } \\ \end{array}\]](/user_files/duweifu/epics/412d6ae795cbffa2d5ce469e158aec4a0158fb36.png "\[\begin{array}{l} L(w,b,\alpha ) = \frac{1}{2}\left\langle {w\cdotw} \right\rangle - \sum\limits_{i = 1}^l {{\alpha _i}[{y_i}\left( {\left\langle {w\cdot{x_i}} \right\rangle + b} \right) - 1]} \\ \quad \quad \quad \quad = \frac{1}{2}\sum\limits_{i,j = 1}^l {{y_i}{y_j}{\alpha _i}{\alpha _j}\left\langle {{x_i},{x_j}} \right\rangle - } \sum\limits_{i,j = 1}^l {{y_i}{y_j}{\alpha _i}{\alpha _j}\left\langle {{x_i},{x_j}} \right\rangle } + \sum\limits_{i = 1}^l {{\alpha _i}} \\ \quad \quad \quad \quad = \sum\limits_{i = 1}^l {{\alpha _i}} - \frac{1}{2}\sum\limits_{i,j = 1}^l {{y_i}{y_j}{\alpha _i}{\alpha _j}\left\langle {{x_i},{x_j}} \right\rangle } \\ \end{array}\]")
根据KKT互补条件，最优解![\[{\alpha ^*}\]](/user_files/duweifu/epics/46e7680e25473d43fd55f07e185d0dab2243f719.png "\[{\alpha ^*}\]")
，必须满足：![\[\alpha _i^*[{y_i}\left( {\left\langle {{w^*}\cdot{x_i}} \right\rangle + {b^*}} \right) - 1] = 0\]](/user_files/duweifu/epics/0446fa92389d789e16a0b8ed32c945edbca94b58.png "\[\alpha _i^*[{y_i}\left( {\left\langle {{w^*}\cdot{x_i}} \right\rangle + {b^*}} \right) - 1] = 0\]")
，这意味着，只有支持平面上的点对应的非零，其余的点对应的为零。这也是支持向量机得名的原因，也为其快速求解铺垫了条件。
，必须满足：，这意味着，只有支持平面上的点对应的非零，其余的点对应的为零。这也是支持向量机得名的原因，也为其快速求解铺垫了条件。
（二）求解
