advanced
Volume 20 Issue 6
December  2023
Turn off MathJax
Article Contents
Relative Articles
Bin Fan, Yuchao Dai, Mingyi He. Rolling Shutter Camera: Modeling, Optimization and Learning. Machine Intelligence Research, vol. 20, no. 6, pp.783-798, 2023. https://doi.org/10.1007/s11633-022-1399-z
Citation: Bin Fan, Yuchao Dai, Mingyi He. Rolling Shutter Camera: Modeling, Optimization and Learning. Machine Intelligence Research, vol. 20, no. 6, pp.783-798, 2023. https://doi.org/10.1007/s11633-022-1399-z
shu

Rolling Shutter Camera: Modeling, Optimization and Learning

doi: 10.1007/s11633-022-1399-z

  • School of Electronics and Information, Northwestern Polytechnical University, Xi′an 710129, China

More Information
  • Author Bio:

    Bin Fan received the B. Sc. degree in statistics and the M. Eng. degree in control science and engineering from Northwestern Polytechnical University, China in 2016 and 2019, respectively. He is currently a Ph. D. degree candiclate in information and communication engineering with School of Electronics and Information, Northwestern Polytechnical University (NPU), China. He was selected to the CVPR 2022 Doctoral Consortium (the only one among Chinese universities). He co-organized the ACCV 2022 tutorial on the topic of rolling shutter cameras. He has published some papers in TPAMI, CVPR, ICCV, TCSVT, CVIU, IVC, etc. His research interests include computer vision, image processing, 3D reconstruction, and deep learning, especially regarding the rolling shutter camera. E-mail: binfan@mail.nwpu.edu.cn ORCID iD: 0000-0002-8028-0166

    Yuchao Dai received the B. Eng., M. Eeg. and Ph. D. degrees all in signal and information processing from Northwestern Polytechnical University, China in 2005, 2008 and 2012, respectively. He is currently a professor with School of Electronics and Information, Northwestern Polytechnical University (NPU), China. He was an ARC DECRA fellow with the research school of engineering at Australian National University, Australia. He won the Best Paper Award in IEEE CVPR 2012, the Best Paper Award Nominee at IEEE CVPR 2020, the DSTO Best Fundamental Contribution to Image Processing Paper Prize at DICTA 2014, the Best Algorithm Prize in NRSFM Challenge at CVPR 2017, the Best Student Paper Prize at DICTA 2017, the Best Deep/Machine Learning Paper Prize at APSIPA ASC 2017. He served as Area Chair in CVPR, ICCV, ACM MM, ACCV, etc. He serves as Publicity Chair in ACCV 2022. His research interests include structure from motion, multi-view geometry, low-level computer vision, deep learning, compressive sensing, and optimization.E-mail: daiyuchao@nwpu.edu.cn (Corresponding author)ORCID iD: 0000-0002-4432-7406

    Mingyi He received the B. Eng. degree in electronic engineering and the M. Eng. degree in signal and systems from Northwestern Polytechnical University (NPU), China in 1982 and 1985, respectively, and the Ph. D. degree in signal and information processing from Xidian University, China in 1994. Since 1985, he has been with School of Electronics and Information, NPU, where he has been a full professor since 1996 and appointed as a chief professor of SIP in 1998. He was the (co)recipient of the 2012 CVPR Best Paper Award, the 2017 APSIPA ASC Best Deep/Machine Learning Paper Award, and the 2017 DICTA Best Student Paper Award. He was a recipient of the Government Lifelong Subsidy from the State Council of China and the Baosteel Outstanding Teacher Award in 2017. He received awards from the IEEE Signal Processing Society in 2014, APSIPA in 2019, China Remote Sensing Committee in 2023, Journal of Image and Graphs in 2022, Signal Processing in 2023, the Chinese Institute of Electronics in 2018 and 2020, and the Shaanxi Institute of Electronics in 2020. He has acted as the general chair or the TPC (co)chair and the area chair for over 30 national and international conferences. He was also an Associate Editor of the IEEE Transactions on Geoscience and Remote Sensing and APSIPA SIP and a Guest Editor of the IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. He is Fellow of CIE and Vice President of APSIPA (2021-2024). His research interests focus on advanced machine vision and intelligent processing, including signal and image processing, computer vision, hyper-spectral remote sensing, 3D information acquisition and processing, and neural network artificial intelligence. E-mail: myhe@nwpu.edu.cn ORCID iD: 0000-0003-2051-6955

  • Received Date: 2022-05-15
  • Accepted Date: 2022-11-29
  • Publish Date: 2023-12-01
    Abstract
  • Most modern consumer-grade cameras are often equipped with a rolling shutter mechanism, which is becoming increasingly important in computer vision, robotics and autonomous driving applications. However, its temporal-dynamic imaging nature leads to the rolling shutter effect that manifests as geometric distortion. Over the years, researchers have made significant progress in developing tractable rolling shutter models, optimization methods, and learning approaches, aiming to remove geometry distortion and improve visual quality. In this survey, we review the recent advances in rolling shutter cameras from two aspects of motion modeling and deep learning. To the best of our knowledge, this is the first comprehensive survey of rolling shutter cameras. In the part of rolling shutter motion modeling and optimization, the principles of various rolling shutter motion models are elaborated and their typical applications are summarized. Then, the applications of deep learning in rolling shutter based image processing are presented. Finally, we conclude this survey with discussions on future research directions.

     

    • 11 https://www.cvl.isy.liu.se/research/datasets/rs-dataset2 https://www.cvl.isy.liu.se/research/datasets/rsba-dataset
    22 https://www.cvl.isy.liu.se/research/datasets/rsba-dataset
    33 https://vision.in.tum.de/data/datasets/rolling-shutter-dataset
    FullText(HTML)
  • loading
    • References(95)
  • [1]
    R. Hartley, A. Zisserman. Multiple View Geometry in Computer Vision, 2nd ed., Cambridge, UK: Cambridge University Press, 2003.
    [2]
    Y. C. Dai, H. D. Li, L. Kneip. Rolling shutter camera relative pose: Generalized epipolar geometry. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, pp. 4132–4140, 2016. DOI: 10.1109/CVPR.2016.448.
    [3]
    C. Albl, Z. Kukelova, T. Pajdla. R6P-rolling shutter absolute pose problem. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, pp. 2292–2300, 2015. DOI: 10.1109/CVPR.2015.7298842.
    [4]
    Y. Z. Lao, O. Ait-Aider. Rolling shutter homography and its applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 8, pp. 2780–2793, 2021. DOI: 10.1109/TPAMI.2020.2977644.
    [5]
    B. Fan, Y. C. Dai. Inverting a rolling shutter camera: Bring rolling shutter images to high framerate global shutter video. In Proceedings of IEEE/CVF International Conference on Computer Vision, IEEE, Montreal, Canada, pp. 4208–4217, 2021. DOI: 10.1109/ICCV48922.2021.00419.
    [6]
    B. B. Zhuang, Q. H. Tran, P. Ji, L. F. Cheong, M. Chandraker. Learning structure-and-motion-aware rolling shutter correction. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Long Beach, USA, pp. 4546–4555, 2019. DOI: 10.1109/CVPR.2019.00468.
    [7]
    Z. H. Zhong, M. D. Cao, X. Sun, Z. R. Wu, Z. Y. Zhou, Y. Q. Zheng, S. Lin, I. Sato. Bringing rolling shutter images alive with dual reversed distortion. In Proceedings of the 17th European Conference on Computer Vision, Springer, Tel Aviv, Israel, pp. 233–249, 2022. DOI: 10.1007/978-3-031-20071-7_14.
    [8]
    Y. Z. Lao, O. Ait-Aider, A. Bartoli. Solving rolling shutter 3D vision problems using analogies with non-rigidity. International Journal of Computer Vision, vol. 129, no. 1, pp. 100–122, 2021. DOI: 10.1007/s11263-020-01368-1.
    [9]
    J. H. Kim, C. Cadena, I. Reid. Direct semi-dense SLAM for rolling shutter cameras. In Proceedings of IEEE International Conference on Robotics and Automation, Stockholm, Sweden, pp. 1308–1315, 2016. DOI: 10.1109/ICRA.2016.7487263.
    [10]
    D. Schubert, N. Demmel, L. von Stumberg, V. Usenko, D. Cremers. Rolling-shutter modelling for direct visual-inertial odometry. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, Macau, China, pp. 2462–2469, 2019. DOI: 10.1109/IROS40897.2019.8968539.
    [11]
    B. B. Zhuang, L. F. Cheong, G. H. Lee. Rolling-shutter-aware differential SfM and image rectification. In Proceedings of IEEE International Conference on Computer Vision, Venice, Italy, pp. 948–956, 2017. DOI: 10.1109/ICCV.2017.108.
    [12]
    O. Saurer, M. Pollefeys, G. H. Lee. Sparse to dense 3D reconstruction from rolling shutter images. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, pp. 3337–3345, 2016. DOI: 10.1109/CVPR.2016.363.
    [13]
    S. Im, H. Ha, G. Choe, H. G. Jeon, K. Joo, I. S. Kweon. Accurate 3D reconstruction from small motion clip for rolling shutter cameras. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 4, pp. 775–787, 2019. DOI: 10.1109/TPAMI.2018.2819679.
    [14]
    J. Hedborg, P. E. Forssen, M. Felsberg, E. Ringaby. Rolling shutter bundle adjustment. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Providence, USA, pp. 1434–1441, 2012. DOI: 10.1109/CVPR.2012.6247831.
    [15]
    M. Meingast, C. Geyer, S. Sastry. Geometric models of rolling-shutter cameras, [Online], Available: https://arxiv.org/abs/0503076, 2005.
    [16]
    B. Fan, Y. C. Dai, M. Y. He. SUNet: Symmetric undistortion network for rolling shutter correction. In Proceedings of IEEE/CVF International Conference on Computer Vision, IEEE, Montreal, Canada, pp. 4521–4530, 2021. DOI: 10.1109/ICCV48922.2021.00450.
    [17]
    O. Saurer, K. Köser, J. Y. Bouguet, M. Pollefeys. Rolling shutter stereo. In Proceedings of IEEE International Conference on Computer Vision, Sydney, Australia, pp. 465–472, 2013. DOI: 10.1109/ICCV.2013.64.
    [18]
    O. Saurer, M. Pollefeys, G. H. Lee. A minimal solution to the rolling shutter pose estimation problem. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, Hamburg, Germany, pp. 1328–1334, 2015. DOI: 10.1109/IROS.2015.7353540.
    [19]
    P. E. Forssén, E. Ringaby. Rectifying rolling shutter video from hand-held devices. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, USA, pp. 507–514, 2010. DOI: 10.1109/CVPR.2010.5540173.
    [20]
    V. Rengarajan, A. N. Rajagopalan, R. Aravind. From bows to arrows: Rolling shutter rectification of urban scenes. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, pp. 2773–2781, 2016. DOI: 10.1109/CVPR.2016.303.
    [21]
    L. Magerand, A. Bartoli, O. Ait-Aider, D. Pizarro. Global optimization of object pose and motion from a single rolling shutter image with automatic 2D-3D matching. In Proceedings of the 12th European Conference on Computer Vision, Springer, Florence, Italy, pp. 456–469, 2012. DOI: 10.1007/978-3-642-33718-5_33.
    [22]
    K. Wang, B. Fan, Y. C. Dai. Relative pose estimation for stereo rolling shutter cameras. In Proceedings of IEEE International Conference on Image Processing, Abu Dhabi, UAE, pp. 463–467, 2020. DOI: 10.1109/ICIP40778.2020.9191254.
    [23]
    O. Ait-Aider, N. Andreff, J. M. Lavest, P. Martinet. Simultaneous object pose and velocity computation using a single view from a rolling shutter camera. In Proceedings of the 9th European Conference on Computer Vision, Springer, Graz, Austria, pp. 56–68, 2006. DOI: 10.1007/11744047_5.
    [24]
    O. Ait-Aider, F. Berry. Structure and kinematics triangulation with a rolling shutter stereo rig. In Proceedings of the 12th International Conference on Computer Vision, IEEE, Kyoto, Japan, pp. 1835–1840, 2009. DOI: 10.1109/ICCV.2009.5459408.
    [25]
    C. Albl, Z. Kukelova, T. Pajdla. Rolling shutter absolute pose problem with known vertical direction. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, pp. 3355–3363, 2016. DOI: 10.1109/CVPR.2016.365.
    [26]
    C. Albl, A. Sugimoto, T. Pajdla. Degeneracies in rolling shutter SfM. In Proceedings of the 14th European Conference on Computer Vision, Springer, Amsterdam, The Netherlands, pp. 36–51, 2016. DOI: 10.1007/978-3-319-46454-1_3.
    [27]
    Z. Kukelova, C. Albl, A. Sugimoto, T. Pajdla. Linear solution to the minimal absolute pose rolling shutter problem. In Proceedings of the 14th Asian Conference on Computer Vision, Springer, Perth, Australia, pp. 265–280, 2019. DOI: 10.1007/978-3-030-20893-6_17.
    [28]
    C. Albl, Z. Kukelova, V. Larsson, T. Pajdla. Rolling shutter camera absolute pose. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 6, pp. 1439–1452, 2020. DOI: 10.1109/TPAMI.2019.2894395.
    [29]
    Z. Kukelova, C. Albl, A. Sugimoto, K. Schindler, T. Pajdla. Minimal rolling shutter absolute pose with unknown focal length and radial distortion. In Proceedings of the 16th European Conference on Computer Vision, Springer, Glasgow, UK, pp. 698–714, 2020. DOI: 10.1007/978-3-030-58558-7_41.
    [30]
    C. Albl, Z. Kukelova, V. Larsson, M. Polic, T. Pajdla, K. Schindler. From two rolling shutters to one global shutter. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Seattle, USA, pp. 2502–2510, 2020. DOI: 10.1109/CVPR42600.2020.00258.
    [31]
    K. Wang, C. H. Liu, K. X. Wang, S. J. Shen. Depth estimation under motion with single pair rolling shutter stereo images. IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 3160–3167, 2021. DOI: 10.1109/LRA.2021.3063695.
    [32]
    E. Ringaby, P. E. Forssén. Efficient video rectification and stabilisation for cell-phones. International Journal of Computer Vision, vol. 96, no. 3, pp. 335–352, 2012. DOI: 10.1007/s11263-011-0465-8.
    [33]
    E. Ito, T. Okatani. Self-calibration-based approach to critical motion sequences of rolling-shutter structure from motion. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, pp. 4512–4520, 2017. DOI: 10.1109/CVPR.2017.480.
    [34]
    Y. Z. Lao, O. Ait-Aider. A robust method for strong rolling shutter effects correction using lines with automatic feature selection. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Salt Lake City, USA, pp. 4795–4803, 2018. DOI: 10.1109/CVPR.2018.00504.
    [35]
    P. Purkait, C. Zach. Minimal solvers for monocular rolling shutter compensation under ackermann motion. In Proceedings of IEEE Winter Conference on Applications of Computer Vision, Lake Tahoe, USA, pp. 903–911, 2018. DOI: 10.1109/WACV.2018.00104.
    [36]
    C. R. Lee, J. H. Yoon, M. G. Park, K. J. Yoon. Gyroscope-aided relative pose estimation for rolling shutter cameras, [Online], Available: https://arxiv.org/abs/1904.06770, 2019.
    [37]
    J. Hedborg, E. Ringaby, P. E. Forssén, M. Felsberg. Structure and motion estimation from rolling shutter video. In Proceedings of IEEE International Conference on Computer Vision Workshops, Barcelona, Spain, pp. 17–23, 2011. DOI: 10.1109/ICCVW.2011.6130217.
    [38]
    B. B. Zhuang, Q. H. Tran. Image stitching and rectification for hand-held cameras. In Proceedings of the 16th European Conference on Computer Vision, Springer, Glasgow, UK, pp. 243–260, 2020. DOI: 10.1007/978-3-030-58571-6_15.
    [39]
    S. Im, H. Ha, G. Choe, H. G. Jeon, K. Joo, I. S. Kweon. High quality structure from small motion for rolling shutter cameras. In Proceedings of IEEE International Conference on Computer Vision, Santiago, Chile, pp. 837–845, 2015. DOI: 10.1109/ICCV.2015.102.
    [40]
    B. Fan, Y. C. Dai, K. Wang. Rolling-shutter-stereo-aware motion estimation and image correction. Computer Vision and Image Understanding, vol. 213, Article number 103296, 2021. DOI: 10.1016/j.cviu.2021.103296.
    [41]
    B. Fan, Y. C. Dai, Z. Y. Zhang, K. Wang. Differential SfM and image correction for a rolling shutter stereo rig. Image and Vision Computing, vol. 124, Article number 104492, 2022. DOI: 10.1016/j.imavis.2022.104492.
    [42]
    Y. Z. Lao, O. Ait-Aider, H. Araujo. Robustified structure from Motion with rolling-shutter camera using straightness constraint. Pattern Recognition Letters, vol. 111, pp. 1–8, 2018. DOI: 10.1016/j.patrec.2018.04.004.
    [43]
    B. Triggs, P. F. McLauchlan, R. I. Hartley, A. W. Fitzgibbon. Bundle adjustment − A modern synthesis. In Proceedings of the International Workshop on Vision Algorithms, Springer, Corfu, Greece, pp. 298–372, 2000. DOI: 10.1007/3-540-44480-7_21.
    [44]
    L. Oth, P. Furgale, L. Kneip, R. Siegwart. Rolling shutter camera calibration. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Portland, USA, pp. 1360–1367, 2013. DOI: 10.1109/CVPR.2013.179.
    [45]
    B. Y. Liao, D. L. Qu, Y. F. Xue, H. Q. Zhang, Y. Z. Lao. Revisiting rolling shutter bundle adjustment: Toward accurate and fast solution, [Online], Available: https://arxiv.org/abs/2209.08503, 2022.
    [46]
    S. Baker, D. Scharstein, J. P. Lewis, S. Roth, M. J. Black, R. Szeliski. A database and evaluation methodology for optical flow. International Journal of Computer Vision, vol. 92, no. 1, pp. 1–31, 2011. DOI: 10.1007/s11263-010-0390-2.
    [47]
    H. C. Longuet-Higgins, K. Prazdny. The interpretation of a moving retinal image. Proceedings of the Royal Society B:Biological Sciences, vol. 208, no. 1173, pp. 385–397, 1980. DOI: 10.1098/rspb.1980.0057.
    [48]
    B. Fan, Y. C. Dai, Z. Y. Zhang, M. Y. He. Fast and robust differential relative pose estimation with radial distortion. IEEE Signal Processing Letters, vol. 29, pp. 294–298, 2022. DOI: 10.1109/LSP.2021.3134593.
    [49]
    V. Rengarajan, Y. Balaji, A. N. Rajagopalan. Unrolling the shutter: CNN to correct motion distortions. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, pp. 2345–2353, 2017. DOI: 10.1109/CVPR.2017.252.
    [50]
    S. C. Su, W. Heidrich. Rolling shutter motion deblurring. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, pp. 1529–1537, 2015. DOI: 10.1109/CVPR.2015.7298760.
    [51]
    P. Purkait, C. Zach, A. Leonardis. Rolling shutter correction in Manhattan world. In Proceedings of IEEE International Conference on Computer Vision, Venice, Italy, pp. 882–890, 2017. DOI: 10.1109/ICCV.2017.101.
    [52]
    A. Patron-Perez, S. Lovegrove, G. Sibley. A spline-based trajectory representation for sensor fusion and rolling shutter cameras. International Journal of Computer Vision, vol. 113, no. 3, pp. 208–219, 2015. DOI: 10.1007/s11263-015-0811-3.
    [53]
    C. Kerl, J. Stückler, D. Cremers. Dense continuous-time tracking and mapping with rolling shutter RGB-D cameras. In Proceedings of IEEE International Conference on Computer Vision, Santiago, Chile, pp. 2264–2272, 2015. DOI: 10.1109/ICCV.2015.261.
    [54]
    F. Bai, A. Sengupta, A. Bartoli. Scanline homographies for rolling-shutter plane absolute pose. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, New Orleans, USA, pp. 8983–8992, 2022. DOI: 10.1109/CVPR52688.2022.00879.
    [55]
    B. Vandeportaele, P. A. Gohard, M. Devy, B. Coudrin. Pose interpolation for rolling shutter cameras using non uniformly time-sampled B-splines. In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, SciTePress, Porto, Portugal, pp. 286–293, 2017. DOI: 10.5220/0006171802860293.
    [56]
    S. Lovegrove, A. Patron-Perez, G. Sibley. Spline fusion: A continuous-time representation for visual-inertial fusion with application to rolling shutter cameras. In Proceedings of the British Machine Vision Conference, BMVA Press, Bristol, UK, pp. 93.1–93.12, 2013. DOI: 10.5244/C.27.93.
    [57]
    J. Mo, J. Islam, J. Sattar. Learning rolling shutter correction from real data without camera motion assumption, [Online], Available: https://arxiv.org/abs/2011.03106, 2020.
    [58]
    J. Z. Huai, Y. K. Zhuang, Y. Lin, G. Jozkow, Q. C. Yuan, D. Chen. Continuous-time spatiotemporal calibration of a rolling shutter camera-IMU system. IEEE Sensors Journal, vol. 22, no. 8, pp. 7920–7930, 2022. DOI: 10.1109/JSEN.2022.3152572.
    [59]
    X. L. Lang, J. J. Lv, J. X. Huang, Y. K. Ma, Y. Liu, X. X. Zuo. Ctrl-VIO: Continuous-time visual-inertial odometry for rolling shutter cameras. IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 11537–11544, 2022. DOI: 10.1109/LRA.2022.3202349.
    [60]
    E. B. Dam, M. Koch, M. Lillholm. Quaternions, Interpolation and Animation, Technical Report DIKU-TR-98/5, Department of Computer Science, University of Copenhagen, Copenhagen, Denmark, 1998.
    [61]
    C. H. Zhao, B. Fan, J. W. Hu, Q. Pan, Z. Xu. Homography-based camera pose estimation with known gravity direction for UAV navigation. Science China Information Sciences, vol. 64, no. 1, Article number 112204, 2021. DOI: 10.1007/s11432-019-2690-0.
    [62]
    C. H. Zhao, B. Fan, J. W. Hu, L. M. Tian, Z. Y. Zhang, S. J. Li, Q. Pan. Pose estimation for multi-camera systems. In Proceedings of IEEE International Conference on Unmanned Systems, Beijing, China, pp. 533–538, 2017. DOI: 10.1109/ICUS.2017.8278403.
    [63]
    S. C. Zhou, R. Yan, J. X. Li, Y. K. Chen, H. J. Tang. A brain-inspired SLAM system based on orb features. International Journal of Automation and Computing, vol. 14, no. 5, pp. 564–575, 2017. DOI: 10.1007/s11633-017-1090-y.
    [64]
    C. L. Wang, T. M. Wang, J. H. Liang, Y. C. Zhang, Y. Zhou. Bearing-only visual SLAM for small unmanned aerial vehicles in GPS-denied environments. International Journal of Automation and Computing, vol. 10, no. 5, pp. 387–396, 2013. DOI: 10.1007/s11633-013-0735-8.
    [65]
    Y. Z. Lao, O. Ait-Aider, A. Bartoli. Rolling shutter pose and ego-motion estimation using shape-from-template. In Proceedings of the 15th European Conference on Computer Vision, Springer, Munich, Germany, pp. 477–492, 2018. DOI: 10.1007/978-3-030-01216-8_29.
    [66]
    O. Ait-Aider, A. Bartoli, N. Andreff. Kinematics from lines in a single rolling shutter image. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, USA, 2007. DOI: 10.1109/CVPR.2007.383119.
    [67]
    R. M. Haralick, D. Lee, K. Ottenburg, M. Nolle. Analysis and solutions of the three point perspective pose estimation problem. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Maui, USA, pp. 592–598, 1991. DOI: 10.1109/CVPR.1991.139759.
    [68]
    M. Panda, B. Das, B. Subudhi, B. B. Pati. A comprehensive review of path planning algorithms for autonomous underwater vehicles. International Journal of Automation and Computing, vol. 17, no. 3, pp. 321–352, 2020. DOI: 10.1007/s11633-019-1204-9.
    [69]
    Y. Yang, F. Qiu, H. Li, L. Zhang, M. L. Wang, M. Y. Fu. Large-scale 3D semantic mapping using stereo vision. International Journal of Automation and Computing, vol. 15, no. 2, pp. 194–206, 2018. DOI: 10.1007/s11633-018-1118-y.
    [70]
    Q. Qi, Q. D. Li, Y. Q. Cheng, Q. Q. Hong. Skeleton marching-based parallel vascular geometry reconstruction using implicit functions. International Journal of Automation and Computing, vol. 17, no. 1, pp. 30–43, 2020. DOI: 10.1007/s11633-019-1189-4.
    [71]
    B. Fan, K. Wang, Y. C. Dai, M. Y. He. RS-DPSNet: Deep plane sweep network for rolling shutter stereo images. IEEE Signal Processing Letters, vol. 28, pp. 1550–1554, 2021. DOI: 10.1109/LSP.2021.3099350.
    [72]
    P. D. Liu, Z. P. Cui, V. Larsson, M. Pollefeys. Deep shutter unrolling network. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Seattle, USA, pp. 5940–5948, 2020. DOI: 10.1109/CVPR42600.2020.00598.
    [73]
    S. Vasu, M. R. M. Mohan, A. N. Rajagopalan. Occlusion-aware rolling shutter rectification of 3D scenes. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Salt Lake City, USA, pp. 636–645, 2018. DOI: 10.1109/CVPR.2018.00073.
    [74]
    H. C. Wu, L. Xiao, Z. H. Wei. Simultaneous video stabilization and rolling shutter removal. IEEE Transactions on Image Processing, vol. 30, pp. 4637–4652, 2021. DOI: 10.1109/TIP.2021.3073865.
    [75]
    Z. H. Zhong, Y. Q. Zheng, I. Sato. Towards rolling shutter correction and deblurring in dynamic scenes. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Nashville, USA, pp. 9215–9224, 2021. DOI: 10.1109/CVPR46437.2021.00910.
    [76]
    K. Praveen, T Lokesh Kumar, A. N. Rajagopalan. Deep network for rolling shutter rectification, [Online], Available: https://arxiv.org/abs/2112.06170, 2021.
    [77]
    B. Fan, Y. C. Dai, Z. Y. Zhang, Q. Liu, M. Y. He. Context-aware video reconstruction for rolling shutter cameras. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, New Orleans, USA, pp. 17551–17561, 2022. DOI: 10.1109/CVPR52688.2022.01705.
    [78]
    B. Fan, Y. C. Dai, H. D. Li. Rolling shutter inversion: Bring rolling shutter images to high framerate global shutter video. IEEE Transactions on Pattern Analysis and Machine Intelligence, to be published. DOI: 10.1109/TPAMI.2022.3212912.
    [79]
    M. D. Cao, Z. H. Zhong, J. H. Wang, Y. Q. Zheng, Y. J. Yang. Learning adaptive warping for real world rolling shutter correction. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, New Orleans, USA, pp. 17764–17772, 2022. DOI: 10.1109/CVPR52688.2022.01726.
    [80]
    C. K. Liang, L. W. Chang, H. H. Chen. Analysis and compensation of rolling shutter effect. IEEE Transactions on Image Processing, vol. 17, no. 8, pp. 1323–1330, 2008. DOI: 10.1109/TIP.2008.925384.
    [81]
    S. Baker, E. Bennett, S. B. Kang, R. Szeliski. Removing rolling shutter wobble. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, USA, pp. 2392–2399, 2010. DOI: 10.1109/CVPR.2010.5539932.
    [82]
    M. Grundmann, V. Kwatra, D. Castro, I. Essa. Calibration-free rolling shutter removal. In Proceedings of IEEE International Conference on Computational Photography, Seattle, USA, pp. 1–8, 2012. DOI: 10.1109/ICCPhot.2012.6215213.
    [83]
    A. Punnappurath, V. Rengarajan, A. N. Rajagopalan. Rolling shutter super-resolution. In Proceedings of IEEE International Conference on Computer Vision, Santiago, Chile, pp. 558–566, 2015. DOI: 10.1109/ICCV.2015.71.
    [84]
    M. Meilland, T. Drummond, A. I. Comport. A unified rolling shutter and motion blur model for 3D visual registration. In Proceedings of IEEE International Conference on Computer Vision, Sydney, Australia, pp. 2016–2023, 2013. DOI: 10.1109/ICCV.2013.252.
    [85]
    J. Park, K. Ko, C. Lee, C. S. Kim. BMBC: Bilateral motion estimation with bilateral cost volume for video interpolation. In Proceedings of the 16th European Conference on Computer Vision, Springer, Glasgow, UK, pp. 109–125, 2020. DOI: 10.1007/978-3-030-58568-6_7.
    [86]
    W. B. Bao, W. S. Lai, C. Ma, X. Y. Zhang, Z. Y. Gao, M. H. Yang. Depth-aware video frame interpolation. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Long Beach, USA, pp. 3698–3707, 2019. DOI: 10.1109/CVPR.2019.00382.
    [87]
    R. Zhang, P. Isola, A. A. Efros, E. Shechtman, O. Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Salt Lake City, USA, pp. 586–595, 2018. DOI: 10.1109/CVPR.2018.00068.
    [88]
    A. Dosovitskiy, G. Ros, F. Codevilla, A. M. López, V. Koltun. CARLA: An open urban driving simulator. In Proceedings of the 1st Annual Conference on Robot Learning, Mountain View, USA, vol. 78, pp. 1–16, 2017.
    [89]
    Z. X. Wang, X. Ji, J. B. Huang, S. Satoh, X. Zhou, Y. Q. Zheng. Neural global shutter: Learn to restore video from a rolling shutter camera with global reset feature. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, New Orleans, USA, pp. 17773–17782, 2022. DOI: 10.1109/CVPR52688.2022.01727.
    [90]
    X. Y. Zhou, P. Q. Duan, Y. Ma, B. X. Shi. EvUnroll: Neuromorphic events based rolling shutter image correction. In Proceedings of IEEE/CVF conference on Computer Vision and Pattern Recognition, New Orleans, USA, pp. 17754–17763, 2022. DOI: 10.1109/CVPR52688.2022.01725.
    [91]
    Y. H. Hu, S. C. Liu, T. Delbruck. V2E: From video frames to realistic DVS events. In Proceedings of IEEE/CVF conference on Computer Vision and Pattern Recognition Workshops, IEEE, Nashville, USA, pp. 1312–1321, 2021. DOI: 10.1109/CVPRW53098.2021.00144.
    [92]
    J. H. Kim, Y. Latif, I. Reid. RRD-SLAM: Radial-distorted rolling-shutter direct SLAM. In Proceedings of IEEE International Conference on Robotics and Automation, Singapore, pp. 5148–5154, 2017. DOI: 10.1109/ICRA.2017.7989602.
    [93]
    D. Schubert, N. Demmel, V. Usenko, J. Stückler, D. Cremers. Direct sparse odometry with rolling shutter. In Proceedings of the 15th European Conference on Computer Vision, Springer, Munich, Germany, pp. 699–714, 2018. DOI: 10.1007/978-3-030-01237-3_42.
    [94]
    J. Mo, J. Islam, J. Sattar. IMU-assisted learning of single-view rolling shutter correction. In Proceedings of the Conference on Robot Learning, London, UK, vol. 164, pp. 861–870, 2021.
    [95]
    S. Tourani, S. Mittal, A. Nagariya, V. Chari, M. Krishna. Rolling shutter and motion blur removal for depth cameras. In Proceedings of IEEE International Conference on Robotics and Automation, Stockholm, Sweden, pp. 5098–5105, 2016. DOI: 10.1109/ICRA.2016.7487715.
    • Relative Articles
    • Supplements(0)
    • Cited By
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(9)  / Tables(5)

    Get Citation
    PDF
    XML
    Entire Issue PDF

    用微信扫码二维码

    分享至好友和朋友圈

    Article Metrics

    Article views (166) PDF downloads(7) Cited by()
    Proportional views
    Related
    Browse MIR
    Published Online Current Issue Special Issue Archive Selected Papers
    About MIR
    Aims and Scopes Index information Editorial Board Contact Us Springer Page

    For Authors
    Guide for Authors Submission Template Copyright Agreement
    For Referees
    Guide for Referees Referees Acknowledgement

    MIR Event
    MIR News MIR Awards
    WeChat: 机器智能研究MIR
    Twitter: MIR_Journal

    © Institute of Automation, Chinese Academy of Sciences. Published by Springer Nature and Science Press. All rights reserved. 京ICP备14019135号-25

    Supported by Beijing Renhe Information Technology Co. Ltd  support: info@rhhz.net  

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return