Revitalizing Art with Technology: A Deep Learning Approach to Virtual Restoration
DOI:
https://doi.org/10.14421/jiska.2025.10.1.87-99Keywords:
Art Restoration, CycleGAN, Deep Learning, PSNR, SSIMAbstract
This study evaluates CycleGAN’s performance in virtual painting restoration, with a focus on color restoration and detail reproduction. We compiled datasets categorized by art style and condition to achieve accurate restorations without altering the original reference materials. Various paintings, including those with a yellow filter, are used to create effective training datasets for CycleGAN. The model utilized cycle consistency loss and advanced data augmentation techniques. We assessed the results using PSNR, SSIM, and Color Inspector metrics, focusing on Claude Monet’s “Nasturtiums in a Blue Vase” and Hermann Corrodi’s “Prayers at Dawn.” The findings demonstrate superior color recovery and preservation of intricate details compared to other methods, confirmed through quantitative and qualitative evaluations. Key contributions include the application of CycleGAN for art restoration, model evaluation, and framework development. Practical implications extend to art conservation, digital library enhancement, art education, and broader access to restored works. Future research may explore dataset expansion, complex architectures, interdisciplinary collaboration, automated evaluation tools, and improved technologies for real-time restoration applications. In conclusion, CycleGAN holds promise for digital art conservation, with ongoing efforts aimed at integrating across fields for effective cultural preservation.
References
Adhikary, A., Bhandari, N., Markou, E., & Sachan, S. (2021). ArtGAN: Artwork Restoration Using Generative Adversarial Networks. 2021 13th International Conference on Advanced Computational Intelligence (ICACI), 199–206. https://doi.org/10.1109/ICACI52617.2021.9435888
Al-Emam, E., Beltran, V., De Meyer, S., Nuyts, G., Wetemans, V., De Wael, K., Caen, J., & Janssens, K. (2021). Removal of a Past Varnish Treatment from a 19th-Century Belgian Wall Painting by Means of a Solvent-Loaded Double Network Hydrogel. Polymers, 13(16), 2651. https://doi.org/10.3390/polym13162651
Engin, D., Genc, A., & Ekenel, H. K. (2018). Cycle-Dehaze: Enhanced CycleGAN for Single Image Dehazing. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 938–9388. https://doi.org/10.1109/CVPRW.2018.00127
Farajzadeh, N., & Hashemzadeh, M. (2021). A Deep Neural Network Based Framework for Restoring the Damaged Persian Pottery via Digital Inpainting. Journal of Computational Science, 56, 101486. https://doi.org/10.1016/j.jocs.2021.101486
Khalid, S., Azad, M. M., Kim, H. S., Yoon, Y., Lee, H., Choi, K.-S., & Yang, Y. (2024). A Review on Traditional and Artificial Intelligence-Based Preservation Techniques for Oil Painting Artworks. Gels, 10(8), 517. https://doi.org/10.3390/gels10080517
Kumar, P., & Gupta, V. (2024). Unpaired Image-to-Image Translation Based Artwork Restoration Using Generative Adversarial Networks. In Smart Innovation, Systems and Technologies (Vol. 372, pp. 581–591). https://doi.org/10.1007/978-981-99-6774-2_52
Li, J., Wang, H., Deng, Z., Pan, M., & Chen, H. (2021). Restoration of Non-Structural Damaged Murals in Shenzhen Bao’an Based on a Generator–Discriminator Network. Heritage Science, 9(1), 6. https://doi.org/10.1186/s40494-020-00478-w
Maali Amiri, M., & Messinger, D. W. (2021). Virtual Cleaning of Works of Art Using Deep Convolutional Neural Networks. Heritage Science, 9(1), 94. https://doi.org/10.1186/s40494-021-00567-4
Maali Amiri, M., & Messinger, D. W. (2023). Virtual Cleaning of Works of Art Using a Deep Generative Network: Spectral Reflectance Estimation. Heritage Science, 11(1), 16. https://doi.org/10.1186/s40494-023-00859-x
Pietroni, E., & Ferdani, D. (2021). Virtual Restoration and Virtual Reconstruction in Cultural Heritage: Terminology, Methodologies, Visual Representation Techniques and Cognitive Models. Information, 12(4), 167. https://doi.org/10.3390/info12040167
Sizyakin, R., Voronin, V. V., & Pizurica, A. (2022). Virtual Restoration of Paintings Based on Deep Learning. In W. Osten, D. Nikolaev, & J. Zhou (Eds.), Fourteenth International Conference on Machine Vision (ICMV 2021) (p. 60). SPIE. https://doi.org/10.1117/12.2624371
Wan, Z., Zhang, B., Chen, D., Zhang, P., Chen, D., Liao, J., & Wen, F. (2020). Bringing Old Photos Back to Life. http://arxiv.org/abs/2004.09484
Wang, H. L., Han, P. H., Chen, Y. M., Chen, K. W., Lin, X., Lee, M. S., & Hung, Y. P. (2018). Dunhuang mural restoration using deep learning. SIGGRAPH Asia 2018 Technical Briefs, 1–4. https://doi.org/10.1145/3283254.3283263
Wang, J., Zhang, E., Cui, S., Wang, J., Zhang, Q., Fan, J., & Peng, J. (2023). GGD-GAN: Gradient-Guided Dual-Branch Adversarial Networks for Relic Sketch Generation. Pattern Recognition, 141, 109586. https://doi.org/10.1016/j.patcog.2023.109586
Wang, S., Cen, Y., Qu, L., Li, G., Chen, Y., & Zhang, L. (2024). Virtual Restoration of Ancient Mold-Damaged Painting Based on 3D Convolutional Neural Network for Hyperspectral Image. Remote Sensing, 16(16), 2882. https://doi.org/10.3390/rs16162882
Wu, Y., Wang, X., Li, Y., Zhang, H., Zhao, X., & Shan, Y. (2021). Towards Vivid and Diverse Image Colorization with Generative Color Prior. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 14357–14366. https://doi.org/10.1109/ICCV48922.2021.01411
Xiao, Y., Jiang, A., Liu, C., & Wang, M. (2019). Single Image Colorization via Modified Cyclegan. 2019 IEEE International Conference on Image Processing (ICIP), 3247–3251. https://doi.org/10.1109/ICIP.2019.8803677
Zeng, Y., Gong, Y., & Zeng, X. (2020). Controllable Digital Restoration of Ancient Paintings Using Convolutional Neural Network and Nearest Neighbor. Pattern Recognition Letters, 133, 158–164. https://doi.org/10.1016/j.patrec.2020.02.033
Zhu, J. Y., Park, T., Isola, P., & Efros, A. A. (2017). Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks. https://doi.org/10.48550/arXiv.1703.10593
Zou, Z., Zhao, P., & Zhao, X. (2021). Virtual Restoration of the Colored Paintings on Weathered Beams in the Forbidden City Using Multiple Deep Learning Algorithms. Advanced Engineering Informatics, 50, 101421. https://doi.org/10.1016/j.aei.2021.101421
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Nurrohmah Endah Putranti, Shyang-Jye Chang, Muhammad Raffiudin
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Authors who publish with this journal agree to the following terms as stated in http://creativecommons.org/licenses/by-nc/4.0
a. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
b. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
c. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
