目的 目前在图像补全领域研究的重点和难点是补全具有复杂结构信息和丰富纹理信息的大破损区域的图像。传统的基于样本块的图像补全算法主要采用规则的模板块和匹配块来进行补全，补全过程中不能充分利用图像的结构或纹理的不规则信息，从而影响算法修复的精度和效率。针对这一问题，本研究提出一种基于不规则块的图像补全算法。方法 在该算法中，首先利用结构稀疏度来区分图像的结构信息和纹理信息并基于结构稀疏度和置信度计算破损区域边界点的优先级，然后选择优先级最高的点构造规则模板块。对处于复杂结构区域的模板块，如果其邻域含有已知的结构信息，则膨胀该规则模板来并利用其周围的结构信息来辅助构造不规则模板块。接下来，在图像完好区域内搜索与该模板块对应的匹配块，如果该匹配块的邻域包含有效的结构信息，则膨胀该匹配块并补充其周围的结构信息来完善该不规则匹配块。最后，利用该不规则匹配块补全破损区域。对于补全过程中块间接缝造成的视觉不连通问题，本研究利用图像的纹理信息来进行修饰。结果 将本文算法与4种修复效果较好的算法(3种基于规则块的算法和1种基于局部敏感哈希的修复算法)进行对比，通过8组经典图像进行实例验证，采用客观评价指标峰值信噪比PSNR和主观视觉连通性进行评价，结果表明本文提出的算法峰值信噪比相较4种对比算法均有0~4dB的提高，且在补全的精细度和视觉连通性方面有更佳的效果。结论 本文提出的算法在补全含有较复杂结构和丰富纹理的破损自然图像、壁画图像和目标物体移除上有较好的修复效果，普适性较强。
The image completion method based on irregular patch
Wang Xinyi,Zhai Donghai(School of Information Science and Technology, Southwest Jiaotong University,School of Information Science and Technology, Southwest Jiaotong University,School of Information Science and Technology, Southwest Jiaotong University,School of Information Science and Technology, Southwest Jiaotong University,School of Information Science and Technology, Southwest Jiaotong University,School of Information Science and Technology, Southwest Jiaotong University,School of Information Science and Technology, Southwest Jiaotong University,School of Information Science and Technology, Southwest Jiaotong University)
The highlight and difficult point of the current research in terms of the image completion is to complete the image of large damaged region with complex structure information and rich texture information. The traditional exemplar-based completion algorithms mainly adapt the regular sample patch and the matching patch to complete damaged region. During the completion process, the irregular information, especially the local irregular information, of the image structure and texture cannot be fully used. This affects the accuracy and efficiency of the algorithm. To solve this problem, the image completion method based on the irregular patch is proposed in this paper. Method In our approach, the structure sparsity is used to distinguish the structure information and texture information of the input image firstly. It means that a patch with a structure sparsity higher than a threshold is marked as a structural patch, and a patch with a structure sparsity lower than a threshold is marked with a texture patch. Besides, the patch priority of pixel on the boundary of damaged region is calculated based on structure sparsity term and confidence term. Then, the pixel of the highest priority is selected as the center of the sample patch. If the neighborhood of a sample patch, which is in the complex structure area of the image, contains the known structure information, this regular patch is dilated in order to construct its corresponding irregular sample patch by using structure information in its neighborhood. After that, the matching patch is being searched in the known part of the image. If the neighborhood of this matching patch contains valid structure information, the matching patch is dilated and the structure information in its neighborhood is supplemented to construct its corresponding irregular matching patch. At last, this irregular matching patch is used to complete its corresponding damaged region. The steps of constructing irregular patches specifically include: transforming the inspection patch into the grayscale patch; using Gaussian blurring to denoise; extracting the structural information by the Canny operator; integrating the structural information and the original image patch to form an irregular mask; picking the mask into the source image to obtain the irregular patch. In terms of the problem of visual dysconnectivity which is caused by seam during the completion process, the texture information of the image is used for modification in the proposed algorithm. The specific method is to map the overlapping region of the texture patches into a network, and then the Graphcut algorithm is used to cut the network in order to generate a new optimal seam. Result Our approach along with four other completion algorithms, which are three algorithms are based on the regular patch and one algorithm is based on sensitive Hash, are used to complete eight classic image benchmarks. The efforts of these algorithms are evaluated by objective and subjective evaluation which are the peak signal-to-noise ratio(PSNR) and the visual connectivity. Compared with those four completion algorithms, the results show that the proposed algorithm can achieve better result in the fineness of the completion as well as the visual connectivity. Besides, the PSNR of the proposed algorithm is improved by 0dB to 4dB. Conclusion The proposed algorithm improves the traditional exemplar-based completion algorithms, which cannot make full use of the image irregularity information, and enhance the accuracy. Simultaneously, the texture information of the image is used to modify visual dysconnectivity so the visual effect of the image is improved. The proposed algorithm has a good application in the restoration of damaged natural images, cultural relics with both geometric structure and rich texture as well as the removal of target objects. Thus, the proposed algorithm has generality.