Image reconstruction using deep learning

Abstract

Fragmented image reconstruction is important in various areas like archaeology, solving 2D puzzle, forensics jobs, etc. Various colors, textures and other non-linear colors based fragmented images can be solved efficiently using deep learning process. To reconstruct fragmented images, it is important to find accurate edges using right boundary and corner detection process. After finding matching edges with proper geometric alignment for fragmented images, deep learning method has been used to understand the shapes and color contexts through classification. Two color contexts have been used to simplify and improve the understanding of the fragmented image context. One is linear pixel color and another is non-linear pixel color of the fragmented images. By considering fragmented images as random fragmented images and many regions of interest, each aligned image pair is passed to deep convolutional neural network (CNN). Here, Faster Region CNN (R-CNN) algorithm has been applied for deep learning process for our proposed methods. In method one (Method-I), we identified linear & non-linear color pixel(s) then apply R-CNN to only non-linear color pixel. Method two (Method-II) doesn’t separate any color, it considers all the fragmented pieces for R-CNN. Both methods have been compared with R-CNN, Fast R-CNN and Faster R-CNN. Visual Geometry Group (VGG) pre-trained base model and PASCAL Visual Object Classes (VOC) dataset have been used for this experiment with various combinations of fragmented images. At the end, calculated the average matching error per 10 sets and the total duration for complete reconstruction of images. Simulations show that the right corner detection with proper pixel classification is very important to get the better accuracy and Faster R-CNN gives faster computation time. For 100 fragmented images, proposed Method-I takes 4-8 mins and 7-11 mins with average matching error (AME) 0.185 and 0.179 respectively. Method-II takes 7-13 mins and 9-15 mins with AME 0.137 and 0.11 respectively.