ImageInpaint：图像修复_image inpainting

前言

传统方法

深度学习

论文研读

今天这篇论文的主角是Image Inpainting via Generative Multi-column Convolutional Neural Networks，相关代码可以参考我的github：Code。

这篇论文提出了generative multi-column network（GMCNN）。摘要如下：

In this paper, we propose a generative multi-column network for image inpainting. This network synthesizes different image components in a parallel manner within one stage. To better characterize global structures, we design a confidence-driven reconstruction loss while an implicit diversified MRF regularization is adopted to enhance local details. The multi-column network combined with the reconstruction and MRF loss propagates local and global information derived from context to the target inpainting regions. Extensive experiments on challenging street view, face, natural objects and scenes manifest that our method produces visual compelling results even without previously common post-processing.

图像复原的概念：

Image inpainting (also known as image completion) aims to estimate suitable pixel information to fill holes in images. It serves various applications such as object removal, image restoration, image denoising, to name a few.

经典的图像复原方法主要定位以下三个重要的问题：

1. 提取合适的特征分析patch之间的相似性
2. 找到近邻的patches
3. 汇总辅助信息
   
   
   
   

方法

该网络主要有三部分：

1. a generator to produce results
2. global and local discriminators for adversarial training
3. a pretrained VGG network to calculate ID-MRF loss

网络结构

• generator
  

• ID-MRF Regularization
  马尔科夫随机场项
  
  

• Information Fusion
  
  

参考

1. Image Inpainting
2. F. Agostinelli, M. R. Anderson, and H. Lee. Adaptive multi-column deep neural networks with application to robust image denoising. In NIPS, pages 1493–1501, 2013
3. D. Ciregan, U. Meier, and J. Schmidhuber. Multi-column deep neural networks for image classification. In CVPR, pages 3642–3649. IEEE, 2012
4. Y. Zhang, D. Zhou, S. Chen, S. Gao, and Y. Ma. Single-image crowd counting via multi-column convolutional neural network. In CVPR, pages 589–597, 2016.