Current Issue

Display Method:
How Good is Google Bard′s Visual Understanding? An Empirical Study on Open Challenges
Haotong Qin , Ge-Peng Ji , Salman Khan , Deng-Ping Fan , Fahad Shahbaz Khan , Luc Van Gool
2023, vol. 20, no. 5,  pp. 605-613, doi: 10.1007/s11633-023-1469-x
Google′s Bard has emerged as a formidable competitor to OpenAI′s ChatGPT in the field of conversational AI. Notably, Bard has recently been updated to handle visual inputs alongside text prompts during conversations. Given Bard′s impressive track record in handling textual inputs, we explore its capabilities in understanding and interpreting visual data (images) conditioned by text questions. This exploration holds the potential to unveil new insights and challenges for Bard and other forthcoming multi-modal Generative models, especially in addressing complex computer vision problems that demand accurate visual and language understanding. Specifically, in this study, we focus on 15 diverse task scenarios encompassing regular, camouflaged, medical, under-water and remote sensing data to comprehensively evaluate Bard′s performance. Our primary finding indicates that Bard still struggles in these vision scenarios, highlighting the significant gap in vision-based understanding that needs to be bridged in future developments. We expect that this empirical study will prove valuable in advancing future models, leading to enhanced capabilities in comprehending and interpreting fine-grained visual data. Our project is released on
A Review and Outlook on Predictive Cruise Control of Vehicles and Typical Applications Under Cloud Control System
Bolin Gao , Keke Wan , Qien Chen , Zhou Wang , Rui Li , Yu Jiang , Run Mei , Yinghui Luo , Keqiang Li
2023, vol. 20, no. 5,  pp. 614-639, doi: 10.1007/s11633-022-1395-3
With the application of mobile communication technology in the automotive industry, intelligent connected vehicles equipped with communication and sensing devices have been rapidly promoted. The road and traffic information perceived by intelligent vehicles has important potential application value, especially for improving the energy-saving and safe-driving of vehicles as well as the efficient operation of traffic. Therefore, a type of vehicle control technology called predictive cruise control (PCC) has become a hot research topic. It fully taps the perceived or predicted environmental information to carry out predictive cruise control of vehicles and improves the comprehensive performance of the vehicle-road system. Most existing reviews focus on the economical driving of vehicles, but few scholars have conducted a comprehensive survey of PCC from theory to the status quo. In this paper, the methods and advances of PCC technologies are reviewed comprehensively by investigating the global literature, and typical applications under a cloud control system (CCS) are proposed. Firstly, the methodology of PCC is generally introduced. Then according to typical scenarios, the PCC-related research is deeply surveyed, including freeway and urban traffic scenarios involving traditional vehicles, new energy vehicles, intelligent vehicles, and multi-vehicle platoons. Finally, the general architecture and three typical applications of the cloud control system (CCS) on PCC are briefly introduced, and the prospect and future trends of PCC are proposed.
Machine Learning Methods in Solving the Boolean Satisfiability Problem
Wenxuan Guo , Hui-Ling Zhen , Xijun Li , Wanqian Luo , Mingxuan Yuan , Yaohui Jin , Junchi Yan
2023, vol. 20, no. 5,  pp. 640-655, doi: 10.1007/s11633-022-1396-2
This paper reviews the recent literature on solving the Boolean satisfiability problem (SAT), an archetypal \begin{document}$ {\cal{NP}}$\end{document}-complete problem, with the aid of machine learning (ML) techniques. Over the last decade, the machine learning society advances rapidly and surpasses human performance on several tasks. This trend also inspires a number of works that apply machine learning methods for SAT solving. In this survey, we examine the evolving ML SAT solvers from naive classifiers with handcrafted features to emerging end-to-end SAT solvers, as well as recent progress on combinations of existing conflict-driven clause learning (CDCL) and local search solvers with machine learning methods. Overall, solving SAT with machine learning is a promising yet challenging research topic. We conclude the limitations of current works and suggest possible future directions. The collected paper list is available at
YuNet: A Tiny Millisecond-level Face Detector
Wei Wu , Hanyang Peng , Shiqi Yu
2023, vol. 20, no. 5,  pp. 656-665, doi: 10.1007/s11633-023-1423-y
Great progress has been made toward accurate face detection in recent years. However, the heavy model and expensive computation costs make it difficult to deploy many detectors on mobile and embedded devices where model size and latency are highly constrained. In this paper, we present a millisecond-level anchor-free face detector, YuNet, which is specifically designed for edge devices. There are several key contributions in improving the efficiency-accuracy trade-off. First, we analyse the influential state-of-the-art face detectors in recent years and summarize the rules to reduce the size of models. Then, a lightweight face detector, YuNet, is introduced. Our detector contains a tiny and efficient feature extraction backbone and a simplified pyramid feature fusion neck. To the best of our knowledge, YuNet has the best trade-off between accuracy and speed. It has only 75 856 parameters and is less than 1/5 of other small-size detectors. In addition, a training strategy is presented for the tiny face detector, and it can effectively train models with the same distribution of the training set. The proposed YuNet achieves 81.1% mAP (single-scale) on the WIDER FACE validation hard track with a high inference efficiency (Intel i7-12700K: 1.6 ms per frame at 320×320). Because of its unique advantages, the repository for YuNet and its predecessors has been popular at GitHub and gained more than 11 K stars at
Effective and Robust Detection of Adversarial Examples via Benford-Fourier Coefficients
Cheng-Cheng Ma , Bao-Yuan Wu , Yan-Bo Fan , Yong Zhang , Zhi-Feng Li
2023, vol. 20, no. 5,  pp. 666-682, doi: 10.1007/s11633-022-1328-1
Adversarial example has been well known as a serious threat to deep neural networks (DNNs). In this work, we study the detection of adversarial examples based on the assumption that the output and internal responses of one DNN model for both adversarial and benign examples follow the generalized Gaussian distribution (GGD) but with different parameters (i.e., shape factor, mean, and variance). GGD is a general distribution family that covers many popular distributions (e.g., Laplacian, Gaussian, or uniform). Therefore, it is more likely to approximate the intrinsic distributions of internal responses than any specific distribution. Besides, since the shape factor is more robust to different databases rather than the other two parameters, we propose to construct discriminative features via the shape factor for adversarial detection, employing the magnitude of Benford-Fourier (MBF) coefficients, which can be easily estimated using responses. Finally, a support vector machine is trained as an adversarial detector leveraging the MBF features. Extensive experiments in terms of image classification demonstrate that the proposed detector is much more effective and robust in detecting adversarial examples of different crafting methods and sources compared to state-of-the-art adversarial detection methods.
FedFV: A Personalized Federated Learning Framework for Finger Vein Authentication
Feng-Zhao Lian , Jun-Duan Huang , Ji-Xin Liu , Guang Chen , Jun-Hong Zhao , Wen-Xiong Kang
2023, vol. 20, no. 5,  pp. 683-696, doi: 10.1007/s11633-022-1341-4
Most finger vein authentication systems suffer from the problem of small sample size. However, the data augmentation can alleviate this problem to a certain extent but did not fundamentally solve the problem of category diversity. So the researchers resort to pre-training or multi-source data joint training methods, but these methods will lead to the problem of user privacy leakage. In view of the above issues, this paper proposes a federated learning-based finger vein authentication framework (FedFV) to solve the problem of small sample size and category diversity while protecting user privacy. Through training under FedFV, each client can share the knowledge learned from its user′s finger vein data with the federated client without causing template leaks. In addition, we further propose an efficient personalized federated aggregation algorithm, named federated weighted proportion reduction (FedWPR), to tackle the problem of non-independent identically distribution caused by client diversity, thus achieving the best performance for each client. To thoroughly evaluate the effectiveness of FedFV, comprehensive experiments are conducted on nine publicly available finger vein datasets. Experimental results show that FedFV can improve the performance of the finger vein authentication system without directly using other client data. To the best of our knowledge, FedFV is the first personalized federated finger vein authentication framework, which has some reference value for subsequent biometric privacy protection research.
ECG Biometrics via Enhanced Correlation and Semantic-rich Embedding
Kui-Kui Wang , Gong-Ping Yang , Lu Yang , Yu-Wen Huang , Yi-Long Yin
2023, vol. 20, no. 5,  pp. 697-706, doi: 10.1007/s11633-022-1345-0
Electrocardiogram (ECG) biometric recognition has gained considerable attention, and various methods have been proposed to facilitate its development. However, one limitation is that the diversity of ECG signals affects the recognition performance. To address this issue, in this paper, we propose a novel ECG biometrics framework based on enhanced correlation and semantic-rich embedding. Firstly, we construct an enhanced correlation between the base feature and latent representation by using only one projection. Secondly, to fully exploit the semantic information, we take both the label and pairwise similarity into consideration to reduce the influence of ECG sample diversity. Furthermore, to solve the objective function, we propose an effective and efficient algorithm for optimization. Finally, extensive experiments are conducted on two benchmark datasets, and the experimental results show the effectiveness of our framework.
Dual-domain and Multiscale Fusion Deep Neural Network for PPG Biometric Recognition
Chun-Ying Liu , Gong-Ping Yang , Yu-Wen Huang , Fu-Xian Huang
2023, vol. 20, no. 5,  pp. 707-715, doi: 10.1007/s11633-022-1366-8
Photoplethysmography (PPG) biometrics have received considerable attention. Although deep learning has achieved good performance for PPG biometrics, several challenges remain open: 1) How to effectively extract the feature fusion representation from time and frequency PPG signals. 2) How to effectively capture a series of PPG signal transition information. 3) How to extract time-varying information from one-dimensional time-frequency sequential data. To address these challenges, we propose a dual-domain and multiscale fusion deep neural network (DMFDNN) for PPG biometric recognition. The DMFDNN is mainly composed of a two-branch deep learning framework for PPG biometrics, which can learn the time-varying and multiscale discriminative features from the time and frequency domains. Meanwhile, we design a multiscale extraction module to capture transition information, which consists of multiple convolution layers with different receptive fields for capturing multiscale transition information. In addition, the dual-domain attention module is proposed to strengthen the domain of greater contributions from time-domain and frequency-domain data for PPG biometrics. Experiments on the four datasets demonstrate that DMFDNN outperforms the state-of-the-art methods for PPG biometrics.
YOLO-CORE: Contour Regression for Efficient Instance Segmentation
Haoliang Liu , Wei Xiong , Yu Zhang
2023, vol. 20, no. 5,  pp. 716-728, doi: 10.1007/s11633-022-1379-3
Instance segmentation has drawn mounting attention due to its significant utility. However, high computational costs have been widely acknowledged in this domain, as the instance mask is generally achieved by pixel-level labeling. In this paper, we present a conceptually efficient contour regression network based on the you only look once (YOLO) architecture named YOLO-CORE for instance segmentation. The mask of the instance is efficiently acquired by explicit and direct contour regression using our designed multi-order constraint consisting of a polar distance loss and a sector loss. Our proposed YOLO-CORE yields impressive segmentation performance in terms of both accuracy and speed. It achieves 57.9% AP@0.5 with 47 FPS (frames per second) on the semantic boundaries dataset (SBD) and 51.1% AP@0.5 with 46 FPS on the COCO dataset. The superior performance achieved by our method with explicit contour regression suggests a new technique line in the YOLO-based image understanding field. Moreover, our instance segmentation design can be flexibly integrated into existing deep detectors with negligible computation cost (65.86 BFLOPs (billion float operations per second) to 66.15 BFLOPs with the YOLOv3 detector).
Transmission Line Insulator Defect Detection Based on Swin Transformer and Context
Yu Xi , Ke Zhou , Ling-Wen Meng , Bo Chen , Hao-Min Chen , Jing-Yi Zhang
2023, vol. 20, no. 5,  pp. 729-740, doi: 10.1007/s11633-022-1355-y
Insulators are important components of power transmission lines. Once a failure occurs, it may cause a large-scale blackout and other hidden dangers. Due to the large image size and complex background, detecting small defect objects is a challenge. We make improvements based on the two-stage network Faster R-convolutional neural networks (CNN). First, we use a hierarchical Swin Transformer with shifted windows as the feature extraction network, instead of ResNet, to extract more discriminative features, and then design the deformable receptive field block to encode global and local context information, which is utilized to capture key clues for detecting objects in complex backgrounds. Finally, the filling data augmentation method is proposed for the problem of insufficient defects and more images of insulator defects under different backgrounds are added to the training set to improve the robustness of the model. As a result, the recall increases from 89.5% to 92.1%, and the average precision increases from 81.0% to 87.1%. To further prove the superiority of the proposed algorithm, we also tested the model on the public data set Pascal visual object classes (VOC), which also yields outstanding results.
Biological Eagle-eye Inspired Target Detection for Unmanned Aerial Vehicles Equipped with a Manipulator
Yi-Min Deng , Si-Yuan Wang
2023, vol. 20, no. 5,  pp. 741-752, doi: 10.1007/s11633-022-1342-3
Inspired by eagle eye mechanisms, the structure and information processing characteristics of the eagle′s visual system are used for the target capture task of an unmanned aerial vehicle (UAV) with a mechanical arm. In this paper, a novel eagle-eye inspired multi-camera sensor and a saliency detection method are proposed. A combined camera system is built by simulating the double fovea structure on the eagle retina. A saliency target detection method based on the eagle midbrain inhibition mechanism is proposed by measuring the static saliency information and dynamic features. Thus, salient targets can be accurately detected through the collaborative work between different cameras of the proposed multi-camera sensor. Experimental results show that the eagle-eye inspired visual system is able to continuously detect targets in outdoor scenes and that the proposed algorithm has a strong inhibitory effect on moving background interference.