Objective

2

Scene text image super-resolution （STISR） is an emerging visual enhancement technology， which is specifically designed to improve the quality of low-resolution text images and the readability of the text. It is extensively applied across various domains such as autonomous driving， document retrieval， and text recognition. The existing STISR methods can be broadly divided into traditional methods and deep learning methods. Traditional methods can enhance images to some extent， but rely on manual feature extractors and complex prior knowledge heavily. On the contrary， deep learning methods showcase significant advantages due to their robust automatic feature extraction and learning abilities. Therefore， deep learning STISR methods have become increasingly popular within the academic community. Recent deep learning based STISR methods begin to utilize rich semantic priors from text images to guide both image reconstruction and text recovery. Unlike methods that solely rely on visual feature extraction， like convolutional neural networks， multilayer perceptrons， and transformers， these emerging methods integrate semantic priors learning with visual feature analysis to achieve more efficient image reconstruction. However， they often neglect the dynamic features of text structure， such as the contextual connections between neighboring characters and directional features. This oversight resulting in the generated text semantic priors cannot be aligned and fused with the image features effectually， thus limiting the quality of the reconstructed images. To overcome these limitations， a new cross modal fusion super-resolution method based on text structure dynamic perception is proposed to enhance both the quality of low-resolution text images and the readability of the text.

Method

2

In this study， we propose an innovative cross modal fusion STISR method based on text structure dynamic perception. Initially， low-resolution text images as input are corrected using a spatial transform network module and a thin plate splines module to adjust the spatial positions of irregular characters， thereby facilitating further processing. These images are then processed through a text structure dynamic perception module and a semantic space alignment module to extract both image modal and text modal features. The text structure dynamic perception module， equipped with a direction sensing block and a context linkage unit， captures multi-scale directional features and deciphers the contextual relationships between character neighborhoods respectively， accurately capturing the dynamic features of the text structure within the image modal. The semantic space alignment module processes the low-resolution images through recognition rendering and binarization to derive text semantic priors and mask priors. These priors are then combined in feature addition to generate advanced text semantic priors， which are aligned with image features under the guidance of the image modal through affine transformations. Lastly， the developed cross modal fusion module employs an adaptive weight distribution strategy to enhance the interactive integration of text and image features across modals， producing the final super-resolved text image.

Result

2

The proposed method was compared against 13 mainstream methods. The evaluation primarily focused on the text recognition accuracy of the reconstructed text images （a critical measure due to the unique nature of text images， where readability and recognizability are paramount）. Secondary metrics included PSNR and SSIM， which， despite their limitations in reflecting image quality due to misalignment of low-resolution and high-resolution images in real datasets， supplemented the evaluation. The experiments on the real dataset Texezoom demonstrated that the proposed method achieved accuracies of 67.1%， 56.6%， and 63.6% on three standard text recognizers ASTER， CRNN， and MORAN respectively， surpassing the existing representative method PERMR by 2.9%， 4.6%， and 3%. And the PSNR and SSIM values of the proposed method are 21.9 and 0.789 respectively， ranking first and second in all compared methods. In addition， the visual comparison of most of these methods further highlighted the superior quality and readability of text images reconstructed by the proposed approach.

Conclusion

2

In this study， we proposed a novel STISR method that effectively guides the generation of advanced text semantic priors by accurately capturing dynamic text structure features. This method promotes the alignment and integration of text semantic priors and image features， significantly enhancing the quality of reconstructed text images and the readability of the text. The experimental results demonstrate that the proposed method holds substantial advantages over other mainstream methods， effectively elevating both the quality of reconstructed text images and the readability of the text.<p>Objective Scene text image super-resolution （STISR） is an emerging visual enhancement technology， which is specifically designed to improve the quality of low-resolution text images and the readability of the text. It is extensively applied across various domains such as autonomous driving， document retrieval， and text recognition. The existing STISR methods can be broadly divided into traditional methods and deep learning methods. Traditional methods can enhance images to some extent， but rely on manual feature extractors and complex prior knowledge heavily. On the contrary， deep learning methods showcase significant advantages due to their robust automatic feature extraction and learning abilities. Therefore， deep learning STISR methods have become increasingly popular within the academic community. Recent deep learning based STISR methods begin to utilize rich semantic priors from text images to guide both image reconstruction and text recovery. Unlike methods that solely rely on visual feature extraction， like convolutional neural networks， multilayer perceptrons， and transformers， these emerging methods integrate semantic priors learning with visual feature analysis to achieve more efficient image reconstruction. However， they often neglect the dynamic features of text structure， such as the contextual connections between neighboring characters and directional features. This oversight resulting in the generated text semantic priors cannot be aligned and fused with the image features effectually， thus limiting the quality of the reconstructed images. To overcome these limitations， a new cross modal fusion super-resolution method based on text structure dynamic perception is proposed to enhance both the quality of low-resolution text images and the readability of the text.</p><p>Method In this study， we propose an innovative cross modal fusion STISR method based on text structure dynamic perception. Initially， low-resolution text images as input are corrected using a spatial transform network module and a thin plate splines module to adjust the spatial positions of irregular characters， thereby facilitating further processing. These images are then processed through a text structure dynamic perception module and a semantic space alignment module to extract both image modal and text modal features. The text structure dynamic perception module， equipped with a direction sensing block and a context linkage unit， captures multi-scale directional features and deciphers the contextual relationships between character neighborhoods respectively， accurately capturing the dynamic features of the text structure within the image modal. The semantic space alignment module processes the low-resolution images through recognition rendering and binarization to derive text semantic priors and mask priors. These priors are then combined in feature addition to generate advanced text semantic priors， which are aligned with image features under the guidance of the image modal through affine transformations. Lastly， the developed cross modal fusion module employs an adaptive weight distribution strategy to enhance the interactive integration of text and image features across modals， producing the final super-resolved text image.</p><p>Result The proposed method was compared against 13 mainstream methods. The evaluation primarily focused on the text recognition accuracy of the reconstructed text images （a critical measure due to the unique nature of text images， where readability and recognizability are paramount）. Secondary metrics included PSNR and SSIM， which， despite their limitations in reflecting image quality due to misalignment of low-resolution and high-resolution images in real datasets， supplemented the evaluation. The experiments on the real dataset Texezoom demonstrated that the proposed method achieved accuracies of 67.1%， 56.6%， and 63.6% on three standard text recognizers ASTER， CRNN， and MORAN respectively， surpassing the existing representative method PERMR by 2.9%， 4.6%， and 3%. And the PSNR and SSIM values of the proposed method are 21.9 and 0.789 respectively， ranking first and second in all compared methods. In addition， the visual comparison of most of these methods further highlighted the superior quality and readability of text images reconstructed by the proposed approach.</p><p>Conclusion In this study， we proposed a novel STISR method that effectively guides the generation of advanced text semantic priors by accurately capturing dynamic text structure features. This method promotes the alignment and integration of text semantic priors and image features， significantly enhancing the quality of reconstructed text images and the readability of the text. The experimental results demonstrate that the proposed method holds substantial advantages over other mainstream methods， effectively elevating both the quality of reconstructed text images and the readability of the text.</p>