Name | Institution | Mail Address | Social Contacts |
---|---|---|---|
Alessandro Bombini | Cultural Heritage Network (CHNet), INFN, Firenze on behalf of European Science Cloud (EOSC) - Pillar | bombini@fi.infn.it | Skype: nickname; Linkedin: https://it.linkedin.com/in/alessandro-bombini-7929a2133; Twitter: nickname ; Hangouts: e-mail; Remove unsed or add other contacts or write "N/A" in case none available |
@authorX | & ... other institutions | authorX e-mail address | Skype: nickname; Linkedin: nickname; Twitter: nickname ; Hangouts: e-mail; Remove unsed or add other contacts |
bombini@fi.infn.it | |
Social | Skype: nickname; Linkedin: nickname; Twitter: nickname ; Hangouts: e-mail; Remove unsed or add other contacts or write "N/A" in case none available |
Jira | add ticketing system endpoint in case you or your team are using one |
ML/DL Technologies | CNN U-Net |
---|---|
Science Fields | Applied Physics |
Difficulty | Entry level |
Language | Eng |
Type | fully annoted, runnable |
Programming Language | Python |
---|---|
ML Toolset | Keras |
Additional libraries | Sci-kit image, PIL, OpenCV, matplotlib |
Suggested Environments | Environment needed to perform use-case. Ex.: INFN-Cloud VM, bare Linux Node, Google CoLab, otther... |
Data Creator | Toronto University |
---|---|
Data Type | RGB Images |
Data Size | 175 MB |
Data Source | https://www.cs.toronto.edu/~kriz/cifar.html |
Inpainting is a conservation process where damaged, deteriorating, or missing parts of an artwork are filled in to present a complete image. This process can be applied to both physical and digital art mediums such as oil or acrylic paintings, chemical photographic prints, 3-dimensional sculptures, or digital images and video.
Here we show a dummy example of Image Inpainting by means of Convolutional Neural Network.
As a first example, we apply a digital damage to a set of 32x32 RGB images; the damage consists in adding a random number of black segments on top of the RGB image (mimicking cuts on the surface).
After that, we train a 2D Convolutional Neural Network of the U-Net type. U-Nets are 2D CNN of the Encoder/Decoder type which are completely symmetrical, i.e. if it has N layers, the k and the N−k layer have the same dimensions, and each layer of the encoding part is forwardly connected either with the subsequent encoding layer and with the symmetrical decoding layer; this means that it can be graphically arranged in a U-shape, hence the name.
We will build a network this form:
0. Input Layer
For the training of the 32x32 case we will use the CIFAR dataset ( https://www.cs.toronto.edu/~kriz/cifar.html ). It consists in 50.000 RGB 32 x 32 images.
We offer also the same model trained over 32x32 images and over 192x192 images, and we use transfer learning to apply the trained NN to different damages.
Since INFN-CHNet (Cultural Heritage Network) is the network of the National Institute of Nuclear Physics (INFN) devoted to cultural heritage, we apply the NN to paintings.
This is a dummy case of Digital Restoration, showing a glimpse of how NN can be applied to Physical Imaging of Cultural Heritages.
The example is available in the GitLab repository:
https://gitlab.com/alessandro.bombini.fi/image-inpainting-tutorial/
You need Jupyter notebook as well as all the Python packages necessary to run it.
The repository is arranged as follows:
Folders:
Notebooks:
Other:
https://gitlab.com/alessandro.bombini.fi/image-inpainting-tutorial/