ΑΙhub.org
 

Using neural networks to “upscale” old films

by
06 February 2020



share this:

The famous short, silent film L’arrivée d’un train en gare de La Ciotat, produced by Auguste and Louis Lumière in 1896, hit the news this week. AI developer Denis Shiryaev used a combination of Gigapixel AI and depth-aware video frame interpolation (DAIN) to “upscale” the film to 4k, 60 frames-per-second quality.

You can watch the upscaled video here:

Here is the original version for comparison:

There were two parts to creating this upscaled video. Firstly, the enhancement to 4k resolution. The algorithm used for this is based on neural networks and was trained with millions of photos. The training process helped to create a sophisticated network that learned the best way to enlarge, enhance, and create natural details.

In addition to the enhanced resolution, Shiryaev utilised DAIN to add frames per second. This video frame interpolation method was developed by Wenbo Bao and colleagues (Shanghai Jiao Tong University, University of California, Merced, and Google) and aims to synthesize new frames in between the original frames. In their arXiv article from April 2019 they propose a novel depth-aware video frame interpolation algorithm which explicitly detects occlusion (when one object in a 3D space is blocking another object from view) using depth information. They developed a depth-aware flow projection layer to synthesize intermediate flows that preferentially sample closer objects rather than those further away. Their algorithm also learns hierarchical features to gather contextual information from neighbouring pixels. The model then warps the input frames, depth maps, and contextual features within an adaptive warping layer. Finally, a frame synthesis network generates the output frame using residual learning.

Shiryaev has also added a colour version which was made using DeOldify. DeOldify was created by Jason Antic and employs Generative Adversarial Networks (GANs) to colorize black and white images.




Lucy Smith , Managing Editor for AIhub.
Lucy Smith , Managing Editor for AIhub.




            AIhub is supported by:


Related posts :



Assured and Trustworthy Human-centered AI – a AAAI Fall symposium

Read some of the highlights from the Assured and Trustworthy Human-centered AI symposium.
08 December 2023, by , and

What’s coming up at #NeurIPS2023?

Find out more about the programme of events, including invited talks, tutorials, workshops, and socials.
07 December 2023, by

Interview with Paula Feldman: generating 3d models of blood vessels

Paula and colleagues have used a recursive neural network approach to better understand, and model, blood vessels.
06 December 2023, by

Experimenting with generative AI in the classroom

Waterloo professor introduces AI in his course and equips students with skills to respond to big problems in tech.
05 December 2023, by

The Wizard of AI – a film by Alan Warburton

Presented at the ODI Data Summit, this video essay addresses the cultural impacts of generative AI.
04 December 2023, by

Forthcoming machine learning and AI seminars: December 2023 edition

A list of free-to-attend AI-related seminars that are scheduled to take place between 1 December 2023 and 31 January 2024.
01 December 2023, by





©2021 - Association for the Understanding of Artificial Intelligence


 












©2021 - ROBOTS Association