浏览全部资源
扫码关注微信
纸质出版日期： 2021-02-16 ，
录用日期： 2020-05-22
移动端阅览
邵绪强, 杨艳, 刘艺林. 流体运动估计光流算法研究综述[J]. 中国图象图形学报, 2021,26(2):355-367.
Xuqiang Shao, Yan Yang, Yilin Liu. Review of optical flow algorithms in fluid motion estimation[J]. Journal of Image and Graphics, 2021,26(2):355-367.
对流体图像序列进行运动分析一直是流体力学、医学和计算机视觉等领域的重要研究课题。从图像对中提取的密集精确的速度矢量场能够为许多领域提供有价值的信息，基于光流法的流体运动估计技术因其独特的优势成为一个有前途的方向。光流法可以获得具有较高分辨率的密集速度矢量场，在小尺度精细结构的测量上有所改进，弥补了基于相关分析法的粒子图像测速技术的不足。此外，光流方法还可以方便的引入各种物理约束，获得较为符合流体运动特性的运动估计结果。为了全面反映基于光流法的流体运动估计算法的研究进展，本文在广泛调研相关文献的基础上，对国内外具有代表性的论文进行了系统阐述。首先介绍了光流法的基本原理，然后将现有算法按照要解决的突出问题进行分类：结合流体力学知识的能量最小化函数，提高对光照变化的鲁棒性，大位移估计和消除异常值。对每类方法，从问题解决过程的角度予以介绍，分析了各类突出问题中现有算法的特点和局限性。最后，总结分析了流体运动估计技术当前面临的问题和挑战，并对未来基于光流法的运动估计算法的研究方向和研究重点进行了展望。
The motion analysis of fluid image sequences has been an important research topic in the fields of fluid mechanics
medicine
and computer vision. The dense and accurate velocity vector field extracted from image pairs can provide valuable information for these fields. For example
in the field of fluid mechanics
the velocity vector field can be used to calculate the divergence and curl fields of fluid; in the field of meteorology
the analysis of the velocity vector field can be used to provide weather forecast; in the field of medicine
the velocity vector field is applied to match medical images. In recent years
fluid motion estimation technology based on an optical flow method has become a promising direction in this subject due to its unique advantages. Compared with particle image velocimetry based on a correlation method
an optical flow method can obtain a denser velocity field and can estimate the motion of a scalar image and not just a particle image. In addition
an optical flow method can easily introduce various physical constraints in accordance with the motion characteristics of the fluid and obtain more accurate motion estimation results. In accordance with the basic principles of an optical flow method
this paper reviews a fluid motion estimation algorithm based on an optical flow method. Referring to a large number of domestic and foreign studies
existing algorithms are classified in accordance with outstanding problems to be solved: combining the energy minimization function with the knowledge of fluid mechanics
improving robustness to illumination changes
estimating large displacements
and eliminating outliers. Combining the minimization function with the knowledge of fluid mechanics introduces various physical constraints for improving the energy minimization function
providing physically meaningful data items and regularization terms
and improving the accuracy of fluid motion estimation results. Algorithms for improving robustness to illumination changes can be classified into four types: using a high-order constancy assumption to expand data items that depend on the constant brightness assumption
extracting illumination-invariant features in the image for data items
using structure-texture decomposition methods
and establishing a mathematical model for light changes. Various methods are applicable to different light change conditions. For the large displacement estimation problem
the pyramid-based multi-resolution optical flow method is first used; however
this method cannot estimate the large displacement of fine structures. To solve this problem
a hybrid motion estimation method that combines the cross-correlation method with a wavelet-based optical flow method is proposed in recent research. This hybrid method uses the cross-correlation method to calculate the large displacement of a fine structure and then uses an optical flow method to refine and redetermine the flow field
combining the advantages of the two methods. The optical flow estimation method based on wavelet transform provides a good mathematical framework for the multi-resolution estimation algorithm and avoids the linear problem that exists in the "coarse-to-fine" multi-resolution framework when estimating large displacements. Methods for eliminating outliers can be divided into three basic categories: methods that use a robust penalty function
median filtering
and forward-backward optical flow consistency check. In this paper
each kind of method is introduced from the perspective of the problem solving process
and the characteristics and limitations of existing algorithms are analyzed in various outstanding problems. Finally
the major research problems are summarized and discussed
and several possible research directions for the future are proposed. First
an optical flow method introduces various physical constraints into the objective function to conform to fluid motion characteristics. Hence
although accurate estimation results can be obtained
the resulting optical flow equation is too complex to solve
and no good numerical solution is obtained. Second
several methods based on an optical flow method exhibit different advantages under varying light change conditions; they also have corresponding shortcomings. Therefore
further research on how to combine the advantages of various methods to cope with different light changing conditions is particularly important. Third
although the hybrid method that combines the cross-correlation and optical flow methods can utilize the advantages of the two methods to obtain high-resolution motion results for the large displacement problem
this method can only be successfully applied to the motion estimation of particle images at present. Thus
exploring this method for other types of fluid motion images is worthwhile. Finally
an optical flow method requires complex variational optimization and its computational efficiency is low. Although some graphics processing unit(GPU) parallel algorithms proposed in recent years have effectively improved computational efficiency
they still cannot achieve real-time estimation. Therefore
improving the computational efficiency of fluid motion estimation algorithms and realizing real-time estimation are among the directions that are worth studying in the future.
流体运动估计光流法流体力学光照变化大位移估计异常值检测
fluid motion estimationoptical flow methodfluid mechanicsillumination changelarge displacement estimationoutlier detection
Adrian R J. 1991. Particle-imaging techniques for experimental fluid mechanics. Annual Review of Fluid Mechanics, 23(1): 261-304[DOI: 10.1146/annurev.fl.23.010191.001401]
Ali S, Daul C, Galbrun E and Blondel W. 2016. Illumination invariant optical flow using neighborhood descriptors. Computer Vision and Image Understanding, 145: 95-110[DOI: 10.1016/j.cviu.2015.12.003]
Arnaud E, Mémin E, Sosa R and Artana G. 2006. A fluid motion estimator for schlieren image velocimetry//Proceedings of the 9th European Conference on Computer Vision. Graz, Austria: Springer: 198-210[DOI: 10.1007/11744023_16http://dx.doi.org/10.1007/11744023_16]
Bailer C, Taetz B and Stricker D. 2015. Flow fields: dense correspondence fields for highly accurate large displacement optical flow estimation//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile: IEEE: 4015-4023[DOI: 10.1109/ICCV.2015.457http://dx.doi.org/10.1109/ICCV.2015.457]
Battiti R, Amaldi E and Koch C. 1991. Computing optical flow across multiple scales: an adaptive coarse-to-fine strategy. International Journal of Computer Vision, 6(2): 133-145[DOI: 10.1007/bf00128153]
Black M J and Anandan P. 1996. The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding, 63(1): 75-104[DOI: 10.1006/cviu.1996.0006]
Brox T, Bruhn A, Papenberg N and Weickert J. 2004. High accuracy optical flow estimation based on a theory for warping//Proceedings of the 8th European Conference on Computer Vision. Prague, Czech Republic: Springer: 25-36[DOI: 10.1007/978-3-540-24673-2_3http://dx.doi.org/10.1007/978-3-540-24673-2_3]
Bruhn A, Weickert J and Schnörr C. 2005. Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods. International Journal of Computer Vision, 61(3): 211-231[DOI: 10.1023/B:VISI.0000045324.43199.43]
Cai S Z, Mémin E, Dérian P and Xu C. 2018. Motion estimation under location uncertainty for turbulent fluid flows. Experiments in Fluids, 59(1): #8[DOI: 10.1007/s00348-017-2458-z]
Cai S Z, Zhou S C, Xu C and Gao Q. 2019. Dense motion estimation of particle images via a convolutional neural network. Experiments in Fluids, 60(4): #73[DOI: 10.1007/s00348-019-2717-2]
Cassisa C, Simoens S, Prinet V and Shao L. 2011. Subgrid scale formulation of optical flow for the study of turbulent flow. Experiments in Fluids, 51(6): 1739-1754[DOI: 10.1007/s00348-011-1180-5]
Charbonnier P, Blanc-Feraud L, Aubert G and Barlaud M.1994. Two deterministic half-quadratic regularization algorithms for computed imaging//Proceedings of the 1st International Conference on Image Processing. Austin, USA: IEEE: 168-172[DOI: 10.1109/ICIP.1994.413553http://dx.doi.org/10.1109/ICIP.1994.413553]
Chen Q F and Koltun V. 2016. Full flow: optical flow estimation by global optimization over regular grids//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 4706-4714[DOI: 10.1109/CVPR.2016.509http://dx.doi.org/10.1109/CVPR.2016.509]
Chen X, Zillé P, Shao L and Corpetti T. 2015. Optical flow for incompressible turbulence motion estimation. Experiments in Fluids, 56(1): #8[DOI: 10.1007/s00348-014-1874-6]
Corpetti T, Heitz D, Arroyo G, Mémin É and Santa-Cruz A. 2006. Fluid experimental flow estimation based on an optical-flow scheme. Experiments in Fluids, 40(1): 80-97[DOI: 10.1007/s00348-005-0048-y]
Corpetti T, MéminÉand Pérez P. 2000. Estimating fluid optical flow//Proceedings of the 15th International Conference on Pattern Recognition. Barcelona, Spain: IEEE: 1033-1036[DOI: 10.1109/ICPR.2000.903722http://dx.doi.org/10.1109/ICPR.2000.903722]
Corpetti T, Mémin É and Pérez P. 2002. Dense estimation of fluid flows. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(3): 365-380[DOI: 10.1109/34.990137]
Corpetti T, MéminÉ, Santa-Cruz A, Heitz D and Arroyo G. 2003. Optical flow estimation in experimental fluid mechanics//Proceedings of the 7th International Symposium on Signal Processing and Its Applications. Paris, France: IEEE: 633-636[DOI: 10.1109/ISSPA.2003.1224783http://dx.doi.org/10.1109/ISSPA.2003.1224783]
Cuzol A, Hellier P and Mémin É. 2007. A low dimensional fluid motion estimator. International Journal of Computer Vision, 75(3): 329-349[DOI: 10.1007/s11263-007-0037-0]
Cuzol A and MéminÉ. 2005. Vortex and source particles for fluid motion estimation//Proceedingsof the 5th International Conference on Scale Space and PDE Methods in Computer Vision. Hofgeismar, Germany: Springer: 254-266[DOI: 10.1007/11408031_22http://dx.doi.org/10.1007/11408031_22]
Demetz O, Hafner D and Weickert J. 2013. The complete rank transform: a tool for accurate and morphologically invariant matching of structures//Proceedings of the British Machine Vision Conference. Bristol, UK: BMVA Press: 50.1-50.12[DOI: 10.5244/C.27.50http://dx.doi.org/10.5244/C.27.50]
Dérian P, Héas P, Herzet C and MéminÉ. 2011. Wavelet-based fluid motion estimation//Proceedings of the 3rd International Conference on Scale Space and Variational Methods in Computer Vision. Ein-Gedi, Israel: Springer: 737-748[DOI: 10.1007/978-3-642-24785-9_62http://dx.doi.org/10.1007/978-3-642-24785-9_62]
Dérian P, Héas P, Herzet C and Mémin É. 2013. Wavelets and optical flow motion estimation. Numerical Mathematics: Theory, Methods and Applications, 6(1): 116-137[DOI: 10.1017/S1004897900001161]
Gadot D and Wolf L. 2016. PatchBatch: a batch augmented loss for optical flow//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE: 4236-4245[DOI: 10.1109/CVPR.2016.459http://dx.doi.org/10.1109/CVPR.2016.459]
Gennert M A and Negahdaripour S. 1987. Relaxing the Brightness Constancy Assumption in Computing Optical Flow. Massachusetts No.AI-MEMO-975. Institute of Tech Cambridge Artificial Intelligence Lab
Hampel F R, Ronchetti E M, Rousseeuw P J and Stahell W A.1986. Robust statistics: the approach based on influence functions.Journal of the Royal Statistical Society: Series D (The Statistician).35(5): 565-566
Heitz D, Héas P, Mémin É and Carlier J. 2008. Dynamic consistent correlation-variational approach for robust optical flow estimation. Experiments in Fluids, 45(4): 595-608[DOI: 10.1007/s00348-008-0567-4]
Heitz D, Mémin É and Schnörr C. 2010. Variational fluid flow measurements from image sequences: synopsis and perspectives. Experiments in Fluids, 48(3): 369-393[DOI: 10.1007/s00348-009-0778-3]
Héas P, Mémin É, Papadakis N and Szantai A É. 2007. Layered estimation of atmospheric mesoscale dynamics from satellite imagery. IEEE Transactions on Geoscience and Remote Sensing, 45(12): 4087-4104[DOI: 10.1109/tgrs.2007.906156]
Horn B K P and Schunck B G. 1981. Determining optical flow. Artificial Intelligence, 17(1/3): 185-203[DOI: 10.1016/0004-3702(81)90024-2]
Jabid T, Kabir M H and Chae O. 2010. Local directional pattern (LDP)—a robust image descriptor for object recognition//Proceedings of the 7th IEEE International Conference on Advanced Video and Signal Based Surveillance. Boston, USA: IEEE: 482-487[DOI: 10.1109/AVSS.2010.17http://dx.doi.org/10.1109/AVSS.2010.17]
Kadri-Harouna S, Dérian P, Héas P and Mémin É. 2013. Divergence-free wavelets and high order regularization. International Journal of Computer Vision, 103(1): 80-99[DOI: 10.1007/s11263-012-0595-7]
Kim Y H, Martínez A M and Kak A C. 2005. Robust motion estimation under varying illumination. Image and Vision Computing, 23(4): 365-375[DOI: 10.1016/j.imavis.2004.05.010]
Kohlberger T, MéminÉand Schnörr C. 2003. Variational dense motion estimation using the Helmholtz decomposition//Proceedings of the 4th International Conference on Scale Space Theories in Computer Vision. Isle of Skye, UK: Springer: 432-448[DOI: 10.1007/3-540-44935-3_30http://dx.doi.org/10.1007/3-540-44935-3_30]
Li X Z, Jia S M and Zhao X. 2012. Robust optical flow estimation for illumination changes//Proceedings of 2012 IEEE International Conference on Robotics and Biomimetics (ROBIO). Guangzhou, China: IEEE: 360-365[DOI: 10.1109/ROBIO.2012.6490993http://dx.doi.org/10.1109/ROBIO.2012.6490993]
Liu T S, Merat A, Makhmalbaf M H M, Fajardo C and Merati P. 2015. Comparison between optical flow and cross-correlation methods for extraction of velocityfields from particle images. Experiments in Fluids, 56(8): #166[DOI: 10.1007/s00348-015-2036-1]
Liu T S and Shen L X. 2008. Fluid flow and optical flow. Journal of Fluid Mechanics, 614: 253-291[DOI: 10.1017/s002211200800 3273]
Liu T S, Salazar D M, Fagehi H, Ghazwani H, Montefort J and Merati P. 2020. Hybrid optical-flow-cross-correlation method for particle image velocimetry. Journal of Fluids Engineering, 142(5): #054501[DOI: 10.1115/1.4045572]
Lu J, Yang H, Zhang Q H and Yin Z P. 2019. A field-segmentation-based variational optical flow method for PIV measurements of nonuniform flows. Experiments in Fluids, 60(9): #142[DOI: 10.1007/s00348-019-2787-1]
Mattavelli M and Nicoulin A. 1994. Motion estimation relaxing the constancy brightness constraint//Proceedings of 1st International Conference on Image Processing. Austin, USA: IEEE: 770-774[DOI: 10.1109/ICIP.1994.413675http://dx.doi.org/10.1109/ICIP.1994.413675]
Menze M, Heipke C and Geiger A. 2015. Discrete optimization for optical flow//Proceedings of the 37th German Conference on Pattern Recognition. Aachen, Germany: Springer: 16-28[DOI: 10.1007/978-3-319-24947-6_2http://dx.doi.org/10.1007/978-3-319-24947-6_2]
Mémin É. 2014. Fluid flow dynamics under location uncertainty. Geophysical and Astrophysical Fluid Dynamics, 108(2): 119-146[DOI: 10.1080/03091929.2013.836190]
Mémin É and Pérez P. 1998. Dense estimation and object-based segmentation of the optical flow with robust techniques. IEEE Transactions on Image Processing, 7(5): 703-719[DOI: 10.1109/83.668027]
Mohamed M A, Rashwan H A, Mertsching B, García M A and Puig D. 2014. Illumination-robust optical flow using a local directional pattern. IEEE Transactions on Circuits and Systems for Video Technology, 24(9): 1499-1508[DOI: 10.1109/tcsvt.2014.2308628]
Müller T, Rabe C, Rannacher J, Franke U and Mester R. 2011. Illumination-robust dense optical flow using census signatures//The 33rd Joint Pattern Recognition Symposium. Frankfurt/Main, Germany: Springer: 236-245[DOI: 10.1007/978-3-642-23123-0_24http://dx.doi.org/10.1007/978-3-642-23123-0_24]
Nakajima Y, Inomata H, Nogawa H, Sato Y, Tamura S, Okazaki K and Torii S. 2003. Physics-based flow estimation of fluids. Pattern Recognition, 36(5): 1203-1212[DOI: 10.1016/s0031-3203(02)00078-x]
Palomares R P, Meinhardt-Llopis E, Ballester C and Haro G. 2017. FALDOI: a new minimization strategy for large displacement variational optical flow. Journal of Mathematical Imaging and Vision, 58(1): 27-46[DOI: 10.1007/s10851-016-0688-y]
Papenberg N, Bruhn A, Brox T, Didas S and Weickert J. 2006. Highly accurate optic flow computation with theoretically justified warping. International Journal of Computer Vision, 67(2): 141-158[DOI: 10.1007/s11263-005-3960-y]
Ranftl R, Bredies K and Pock T. 2014. Non-local total generalized variation for optical flow estimation//Proceedings of the 13th European Conference on Computer Vision. Zurich, Switzerland: Springer: 439-454[DOI: 10.1007/978-3-319-10590-1_29http://dx.doi.org/10.1007/978-3-319-10590-1_29]
Rashwan H A, Mohamed M A, García M A, Mertsching B and Puig D. 2013. Illumination robust optical flow model based on histogram of oriented gradients//Proceedings of the 35th German Conference on Pattern Recognition. Berlin, Germany: Springer: 354-363[DOI: 10.1007/978-3-642-40602-7_38http://dx.doi.org/10.1007/978-3-642-40602-7_38]
Roth S, Lempitsky V and Rother C. 2009. Discrete-continuous optimization for optical flow estimation//Cremers D, Rosenhahn B, Yuille A L and Schmidt F R, eds. Statistical and Geometrical Approaches to Visual Motion Analysis. Berlin, Heidelberg: Springer: 1-22[DOI: 10.1007/978-3-642-03061-1_1http://dx.doi.org/10.1007/978-3-642-03061-1_1]
Ruhnau P, Stahl A and Schnörr C. 2007. Variational estimation of experimental fluid flows with physics-based spatio-temporal regularization. Measurement Science and Technology, 18(3): 755-763[DOI: 10.1088/0957-0233/18/3/027]
Schmidt B E and Sutton J A. 2019. High-resolution velocimetry from tracer particle fields using a wavelet-based optical flow method. Experiments in Fluids, 60(3): #37[DOI: 10.1007/s00348-019-2685-6]
Schmidt B E and Sutton J A. 2020. Improvements in the accuracy of wavelet-based optical flow velocimetry (wOFV) using an efficient and physically based implementation of velocity regularization. Experiments in Fluids, 61(2): #32[DOI: 10.1007/s00348-019-2869-0]
Seong J H, Song M S, Nunez D, Manera A and Kim E S. 2019. Velocity refinement of PIV using global optical flow. Experiments in Fluids, 60(11): #174[DOI: 10.1007/s00348-019-2820-4]
Sun D Q, Roth S and Black M J. 2010. Secrets of optical flow estimation and their principles//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, USA: IEEE: 2432-2439[DOI: 10.1109/CVPR.2010.5539939http://dx.doi.org/10.1109/CVPR.2010.5539939]
Sun D Q, Roth S and Black M J. 2014. A quantitative analysis of current practices in optical flow estimation and the principles behind them. International Journal of Computer Vision, 106(2): 115-137[DOI: 10.1007/s11263-013-0644-x]
Sun D Q, Roth S, Lewis J P and Black M J. 2008. Learning optical flow//Proceedings of the 10th European Conference on Computer Vision. Marseille, France: Springer: 83-97[DOI: 10.1007/978-3-540-88690-7_7http://dx.doi.org/10.1007/978-3-540-88690-7_7]
Uras S, Girosi F, Verri Aand Torre V. 1988. A computational approach to motion perception. Biological Cybernetics, 60(2): 79-87[DOI: 10.1007/bf00202895]
Wedel A, Pock T, Zach C, Bischof H and Cremers D. 2009. An improved algorithm for TV-L1optical flow//Cremers D, Rosenhahn B, Yuille A L and Schmidt F R, eds. Statistical and Geometrical Approaches to Visual Motion Analysis. Berlin, Germany: Springer: 23-45[DOI: 10.1007/978-3-642-03061-1_2http://dx.doi.org/10.1007/978-3-642-03061-1_2].
Wildes R P, Amabile M J, Lanzillotto A M and Leu T S. 2000. Recovering estimates of fluid flow from image sequence data. Computer Vision and Image Understanding, 80(2): 246-266[DOI: 10.1006/cviu.2000.0874]
Xu S Y, Zhuang H, Fu X, Zhou J L and Chen M S. 2017. GPU-based fluid motion estimation using energy constraint. Journal of Circuits, Systems and Computers, 26(2): #1750022[DOI: 10.1142/S0218126617500220]
Yang Z F and Johnson M. 2017. Hybrid particle image velocimetry with the combination of cross-correlation and optical flow method. Journal of Visualization, 20(3): 625-638[DOI: 10.1007/s12650-017-0417-7]
Yuan J Y, Qiong W and Li B L. 2014. Robust optical flow computation under varying illumination using rank transform. Sensors and Transducers, 169(4): 84-90
Yuan J Y, Wang Q and Li B L. 2015. Using image texture and improved census transformation to improve the robustness of variational optical flow against illumination changes. Semiconductor Optoelectronics, 36(4): 642-649
袁建英, 王琼, 李柏林. 2015.光照变化条件下结合图像纹理与改进census变换的鲁棒光流算法.半导体光电, 36(4): 642-649
Zabih R and Woodfill J. 1994. Non-parametric local transforms for computing visual correspondence//Proceedings of the 3rd European Conference on Computer Vision. Stockholm, Sweden: Springer: 151-158[DOI: 10.1007/BFb0028345http://dx.doi.org/10.1007/BFb0028345]
Zhong Q L, Yang H and Yin Z P. 2017. An optical flow algorithm based on gradient constancy assumption for PIV image processing. Measurement Science and Technology, 28(5): #055208[DOI: 10.1088/1361-6501/aa6511]
Zhou L, Kambhamettu C and Goldgof D B. 2000. Fluid structure and motion analysis from multi-spectrum 2D cloud image sequences//Proceedings of 2000 IEEE Conference on Computer Vision and Pattern Recognition. Hilton Head Island, USA: IEEE: 744-751[DOI: 10.1109/CVPR.2000.854949http://dx.doi.org/10.1109/CVPR.2000.854949]
Zille P, Corpetti T, Shao L and Chen X. 2014. Observation model based on scale interactions for optical flow estimation. IEEE Transactions on Image Processing, 23(8): 3281-3293[DOI: 10.1109/TIP.2014.2328893]
Zuo Q and Qi Y. 2017. A novel spatial-temporal optical flow method for estimating the velocity fields of a fluid sequence. The Visual Computer, 33(3): 293-302[DOI: 10.1007/s00371-015-1195-7]
相关作者
相关机构
京公网安备11010802024621