LSSVM prediction based on the improved least square support vector machine LSSVM prediction algorithm

1) Optimization objectives

2) Lagrange multiplier method

Where α I \alpha_iα I is the Lagrange multiplier and also the support values 3) to solve the optimization conditions

4) Solve the duality problem (the same as SVM does not do any calculation for W, Ww and E ee)

LLSVM obtains the values of optimization variables A AA and B BB by solving the above linear equations, which is simpler than QP problem

5) Differences with standard SVM

A. Use equality constraints instead of inequality constraints; B. Since equality constraint is applied to each sample point, no constraint is imposed on the relaxed vector, which is also an important reason why LSSVM loses sparsity; C. The problem is further simplified by solving equality constraints and least squares problems.

1) Problem description

2) Optimization objectives

3) Solve the duality problem (the same as SVM does not do any calculation for W, Ww and E ee)

LLSVM obtains the values of optimization variables A AA and B BB by solving the above linear equations, which is simpler than QP problem

Note that when solving the classification task, we get α I =γe I \alpha_{I}=\gamma{e_{I}}α I =γei, Because the Lagrangian ≠ 0 \alpha_{I}\ NEq {0}α I ≠ 0 in the Equation constraint in the Lagrangian multiplier method, all training samples are treated as support vectors, which leads to the loss of the original sparse nature of SVM. However, training set can be thinned by “pruning” based on support, which can also be regarded as a kind of sparse approximate operation.

Moth-flame optimization (MFO), proposed by Seyedali Mirjalili in 2015, provides a new heuristic search paradigm for optimization: spiral search.

Moths have a special way of navigating at night: laterally. It flies at an Angle to the moon to maintain a straight path, but this only works if the light is far away from the moth. When the artificial light was present, the moth was fooled by the artificial light and kept flying at the same Angle as the artificial light. Because it was too close to the light, its flight path was not a straight line, but a spiral path.

Inspired by this natural phenomenon, Seyedali Mirjalili abstracted the process of the moth spiraling around the light source into an optimization process. The whole space of the moth’s flight is the solution space of the problem. A moth is a solution of the problem, and the flame (light source) is an optimal solution of the problem. The algorithm can avoid falling into local optimum. When there are enough moths and flames, the moths can search most of the solution space, so as to ensure the exploration ability of the algorithm. In the process of optimization, the flame number decreases with the increase of the number of iterations, which enables moths to fully search the neighborhood space of the better solution and ensures the utilization of the algorithm.

It is based on the above characteristics that MFO finds a balance between exploration and utilization, so that the algorithm has a better effect in the optimization problem.

In general, MFO is also a random heuristic search algorithm based on population. The biggest difference between MFO and PSO, GSA and other algorithms is that its particle search path is spiral, and particles move around the optimal solution in a spiral way instead of straight line.

The process of MFO is as follows: 1. Initialization of moth population 2. Fitness evaluation of moth population 3. The following process is repeated until the stop standard is reached: 3.1 Adaptive updating of flame number N, when the number of iterations is 1, the number of moths is the number of flame 3.2 Moth population fitness is sorted, and n moths with better fitness are taken as flame 3.3 updating search parameters of moths. 3.4 Update the position of the moth according to the flame and flight parameters corresponding to each moth

Moth-flame optimization algorithm: A novel Nature-inspired heuristic paradigm is the exact formula for updating the location of moths

% = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = % initialization CLC close all clear format long tic % = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = % % import data data = xlsread (' 1. XLSX '); [row,col]=size(data); x=data(:,1:col-1); y=data(:,col); set=1; % Set measurement sample number row1=row-set; % train_x=x(1:row1,:); train_y=y(1:row1,:); test_x=x(row1+1:row,:); Test_y =y(row1+1:row,:); % forecast output train_x=train_x'; train_y=train_y'; test_x=test_x'; test_y=test_y'; %% data normalization [train_x, MINx, MAxx, train_YY, MINy, MAxy] =premnmx(train_x,train_y); test_x=tramnmx(test_x,minx,maxx); train_x=train_x'; train_yy=train_yy'; train_y=train_y'; test_x=test_x'; test_y=test_y'; %% parameter initialization EPS = 10^(-6); %% define LSSVM parameter type='f'; kernel = 'RBF_kernel'; N=20; % Number of search agents Max_iteration=100; % Maximum numbef of iterations Leader_pos=zeros(1,dim); Leader_score=inf; %change this to -inf for maximization problems %Initialize the positions of search agents % for i=1:SearchAgents_no % Positions(i,1)=ceil(rand(1)*(ub(1)-lb(1))+lb(1)); % Positions(i,2)=ceil(rand(1)*(ub(2)-lb(2))+lb(2)); Plot (Convergence_curve,'LineWidth',2); The title ([' puhuo optimization algorithm fitness curves', '(parameters c1 =', num2str (Best_flame_pos (1)), ', c2 = ', num2str (Best_flame_pos (2)), ', termination of algebra = ', num2str (Max_iterat ion),')'],'FontSize',13); Xlabel (' Evolution algebra '); Ylabel (' error fitness '); bestc = Best_flame_pos(1); bestg = Best_flame_pos(2); gam=bestc; sig2=bestg; model=initlssvm(train_x,train_yy,type,gam,sig2,kernel,proprecess); Trainlssvm (model); Train_predict_y, zT,model = SIMlSSVM (model,train_x); [test_predict_y,zt,model]=simlssvm(model,test_x); Predict_y =postmnmx(train_predict_Y,miny,maxy); % test_predict=postmnmx(test_predict_y,miny,maxy); % mean square error trainmse=sum(train_predict-train_y).^2)/length(train_y); %testmse=sum((test_predict-test_y).^2)/length(test_y) for i=1:set RD(i)=(test_predict(i)-test_y(i))/test_y(i)*100; end for i=1:set D(i)=test_predict(i)-test_y(i); Figure (train_predict,': *') hold on plot(train_y,'- *') end RD=RD' disp(' train_predict,': *') Legend (' forecast output ',' expected output ') title(' Optimized SVM network forecast output ','fontsize',12) ylabel(' function output ','fontsize',12) xlabel(' sample ','fontsize',12) TOC % Calculation timeCopy the code