A list,
It is a hot topic in the field of computer vision to determine the relative motion parameters between objective objects and observers based on the analysis of dynamic images and time-varying image sequences. In the medical, industrial, national defense and other aspects have very important practical significance. This paper will introduce the use of optical flow model to analyze motion graphics: according to the conservation principle of image pixel intensity, establish the optical flow constraint equation, calculate motion parameters, and finally calculate the horizontal and vertical displacement between two frames of sample images with examples, and draw optical flow vector map.
1 Important concept: Optical flow Optical flow is the instantaneous velocity field of the motion of a space moving object at the pixel point on the observed surface, which contains the relative motion relationship between the object and the imaging sensor system.
2. Optical flow constraint equation
Objects generally move relatively continuously in space, so the images projected onto the retina/sensor plane should also change continuously. Therefore, it can be assumed that the moving image function ƒ(x,y,t) is a continuous function of variables x,y and t.
Therefore, we can believe that the intensity of image points at t moment and t+dt moment is equal:
Then, Taylor series expansion is performed on the right side of the above equation in (x,y,t), and the higher-order terms are removed and divided by dt at the same time, which can be obtained:
This is the optical flow constraint equation.
This expression represents: the time change rate of image intensity is equal to the product of the space change rate of image intensity and the speed of motion (written in vector form is more intuitive).
Reporter:
Substituted into (2), the second form of the optical flow constraint equation can be obtained:
3. Applicability analysis of optical flow constraint equation
3.1 In practical application, there may be points of intensity discontinuity in the image. In this case, as long as the image intensity changes uniformly around the discontinuous region, the optical flow constraint equation still holds.
3.2 It is necessary to ensure that the change of every point on the image plane is completely caused by the relative motion between the object and the observer.
4. Solution of optical flow constraint equation
The solution of optical flow constraint equation is to calculate two unknown velocity components U and V.
According to Equation (3), the solutions of u and v are constrained on a straight line in space. Without another constraint condition, we cannot uniquely determine the solutions of u and v. So a new constraint must be added.
Obviously, the change of U and V with pixel point movement is slow, and the change of local area is small, especially when the target is moving without deformation rigid body, the spatial change rate of local area velocity is 0.
Let UAVG and VAVG represent the average speed in the field of U and V respectively. According to the above analysis, it can be known that:
This constraint equation reflects the necessary conditions that the velocity change rate should meet, and the optical flow constraint equation reflects the relative relationship between the velocity and the gray change rate.
Combined with the optical flow constraint equation and constraint conditions, the following minimization equation can be established for all pixels:
Where ε is the total error of the minimization problem and λ is the Lagrange multiplier with additional constraints.
To minimize ε, take the derivative of ε with respect to u and v and take the derivative to be 0, and get:
Can be derived:
The above equation can be solved by iterative method to obtain u and v.
In practical solution, the partial derivatives of image intensity in space and time Ex, Ey and Et are also needed to be calculated. ƒ I, j, and K are the intensity of the image at (x,y, T), and ƒ I, j, and K +1 are the intensity of the image at (x,y, T +dt). Then Ex, Ey, and Et can be approximately calculated according to the following formula:
For all x and y values, the iterative calculation ends when the results of the iterative operation meet the prespecified estimated tolerances ε1 and ε2, i.e. :
5 Actual analysis procedure
5.1 Initialization: Number of iterations n = 1, λ = 1, u(n-1) = V (n-1) = 0.
5.2 Calculate partial derivatives Ex, Ey, Et of image intensity according to Formula (9).
5.3 Calculate u(n) and v(n) according to Formula (8).
5.4 Determine whether tolerance condition (10) is met. If so, the iteration ends, and u(n) and V (n) are the desired target values.
5.5 Determine whether n <= the maximum number of iterations is satisfied. If so, the iteration ends, and u(n) and V (n) are the desired target values. Otherwise, the number of iterations +1, go to 3 execution.
6 Flexibility of optical flow Method Through the solution process demonstrated above, it can be seen that optical flow method has very good flexibility in algorithm implementation, mainly reflected in: 6.1 Calculation accuracy and iterative convergence speed can be controlled by adjusting the value of λ. 6.2 Motion parameters can be calculated only by calculating partial molecular blocks in the image.
Ii. Source code
function varargout = optical_flow(varargin)
% OPTICAL_FLOW M-file for optical_flow.fig
% OPTICAL_FLOW, by itself, creates a new OPTICAL_FLOW or raises the existing
% singleton*.
%
% H = OPTICAL_FLOW returns the handle to a new OPTICAL_FLOW or the handle to
% the existing singleton*.
%
% OPTICAL_FLOW('CALLBACK',hObject,eventData,handles,...) calls the local
% function named CALLBACK in OPTICAL_FLOW.M with the given input arguments.
%
% OPTICAL_FLOW('Property'.'Value',...). creates anew OPTICAL_FLOW or raises the
% existing singleton*. Starting from the left, property value pairs are
% applied to the GUI before optical_flow_OpeningFunction gets called. An
% unrecognized property name orinvalid value makes property application % stop. All inputs are passed to optical_flow_OpeningFcn via varargin. % % *See GUI Options on GUIDE's Tools menu. Choose "GUI allows only one % instance to run (singleton)".
%
% See also: GUIDE, GUIDATA, GUIHANDLES
% Copyright 2002- 2003. The MathWorks, Inc.
% Edit the above text to modify the response to help optical_flow
% Last Modified by GUIDE v2. 5 23-Jul- 2012. 15:10:21
% Begin initialization code - DO NOT EDIT
gui_Singleton = 1;
gui_State = struct('gui_Name', mfilename, ...
'gui_Singleton', gui_Singleton, ...
'gui_OpeningFcn', @optical_flow_OpeningFcn, ...
'gui_OutputFcn', @optical_flow_OutputFcn, ...
'gui_LayoutFcn', [],...'gui_Callback'[]);if nargin && ischar(varargin{1})
gui_State.gui_Callback = str2func(varargin{1});
end
if nargout
[varargout{1:nargout}] = gui_mainfcn(gui_State, varargin{:});
else
gui_mainfcn(gui_State, varargin{:});
end
% End initialization code - DO NOT EDIT
% --- Executes just before optical_flow is made visible.
function optical_flow_OpeningFcn(hObject, eventdata, handles, varargin)
% This function has no output args, see OutputFcn.
% hObject handle to figure
% eventdata reserved - to be defined in a future version of MATLAB
% handles structure with handles and user data (see GUIDATA)
% varargin command line arguments to optical_flow (see VARARGIN)
% Choose default command line output for optical_flow
handles.output = hObject;
% Update handles structure
guidata(hObject, handles);
% UIWAIT makes optical_flow wait for user response (see UIRESUME)
% uiwait(handles.figure1);
% --- Outputs from this function are returned to the command line.
function varargout = optical_flow_OutputFcn(hObject, eventdata, handles)
% varargout cell array for returning output args (see VARARGOUT);
% hObject handle to figure
% eventdata reserved - to be defined in a future version of MATLAB
% handles structure with handles and user data (see GUIDATA)
% Get default command line output from handles structure
varargout{1} = handles.output;
% --- Executes on button press in pushbutton_start.
function pushbutton_start_Callback(hObject, eventdata, handles)
% hObject handle to pushbutton_start (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB
% handles structure with handles and user data (see GUIDATA)
global FilePath1
global FilePath2
global version
if ( isequal(FilePath1,'No Picture') || isequal(FilePath2,'No Picture'))
errordlg('Error selecting picture! '.'MATLAB error');
return
end
switch version
case { 0.1.2 }
optic_flow_brox(FilePath1, FilePath2);
case 3
Horn_WJY(FilePath2, FilePath1);
case 4
Lucas_Kanade(FilePath2, FilePath1);
otherwise
errordlg('Selection algorithm error! '.'MATLAB error');
end
% --- Executes on button press in pushbutton_pic1.
function pushbutton_pic1_Callback(hObject, eventdata, handles)
% hObject handle to pushbutton_pic1 (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB
% handles structure with handles and user data (see GUIDATA)
global FilePath1
[FileName,PathName] = uigetfile({'*.jpg';'*.bmp'},'Select the picture');
if ~isequal(FileName,0)
FilePath1 = fullfile(PathName,FileName);
set(handles.edit_lj1,'String',FilePath1);
img=imread(FilePath1);
axes(handles.axes_pic1);
imshow(img);
end
guidata(hObject, handles);
% --- Executes on button press in pushbutton_pic2.
function pushbutton_pic2_Callback(hObject, eventdata, handles)
% hObject handle to pushbutton_pic2 (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB
% handles structure with handles and user data (see GUIDATA)
global FilePath2
[FileName,PathName] = uigetfile({'*.jpg';'*.bmp'},'Select the picture');
if ~isequal(FileName,0)
FilePath2 = fullfile(PathName,FileName);
set(handles.edit_lj2,'String',FilePath2);
img=imread(FilePath2);
axes(handles.axes_pic2);
imshow(img);
end
guidata(hObject, handles);
function edit_lj1_Callback(hObject, eventdata, handles)
% hObject handle to edit_lj1 (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB
% handles structure with handles and user data (see GUIDATA)
% Hints: get(hObject,'String') returns contents of edit_lj1 as text
% str2double(get(hObject,'String')) returns contents of edit_lj1 as a double
% --- Executes during object creation, after setting all properties.
function edit_lj1_CreateFcn(hObject, eventdata, handles)
% hObject handle to edit_lj1 (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB
% handles empty - handles not created until after all CreateFcns called
% Hint: edit controls usually have a white background on Windows.
% See ISPC and COMPUTER.
if ispc
set(hObject,'BackgroundColor'.'white');
else
set(hObject,'BackgroundColor',get(0.'defaultUicontrolBackgroundColor'));
end
function edit_lj2_Callback(hObject, eventdata, handles)
% hObject handle to edit_lj2 (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB
% handles structure with handles and user data (see GUIDATA)
% Hints: get(hObject,'String') returns contents of edit_lj2 as text
% str2double(get(hObject,'String')) returns contents of edit_lj2 as a double
% --- Executes during object creation, after setting all properties.
function edit_lj2_CreateFcn(hObject, eventdata, handles)
% hObject handle to edit_lj2 (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB
% handles empty - handles not created until after all CreateFcns called
% Hint: edit controls usually have a white background on Windows.
% See ISPC and COMPUTER.
if ispc
set(hObject,'BackgroundColor'.'white');
else
set(hObject,'BackgroundColor',get(0.'defaultUicontrolBackgroundColor'));
end
function edit_arithmetic_Callback(hObject, eventdata, handles)
% hObject handle to edit_arithmetic (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB
% handles structure with handles and user data (see GUIDATA)
% Hints: get(hObject,'String') returns contents of edit_arithmetic as text
% str2double(get(hObject,'String')) returns contents of edit_arithmetic as a double
% --- Executes during object creation, after setting all properties.
function edit_arithmetic_CreateFcn(hObject, eventdata, handles)
% hObject handle to edit_arithmetic (see GCBO)
% eventdata reserved - to be defined in a future version of MATLAB
% handles empty - handles not created until after all CreateFcns called
% Hint: edit controls usually have a white background on Windows.
% See ISPC and COMPUTER.
if ispc
set(hObject,'BackgroundColor'.'white');
else
set(hObject,'BackgroundColor',get(0.'defaultUicontrolBackgroundColor'));
end
Copy the code
3. Operation results
Fourth, note
Version: 2014 a