New Computing technique generates comic books from movies, videos

Computing algorithms can be used for business operations across industries is now widely acceptable. With consistent effort, computer scientists have developed computational techniques that can generate texts, images, and other types of data automatically. In fact, the newly developed computational techniques are highly advantageous, in particular, to create data or creative networks that are time-intensive and demanding to be done manually.

In a new development, a team of researchers at Dalian University of Technology in City University of Hong Kong and China have created an innovative framework that can automatically produce manga comic books. Typically, these comics are designed by highly skilled professional artists and are labor-intensive. Elaborately, the framework creates comic books by pulling out data from TV series, animations, movies, or other videos.

“Importantly, the framework proposed is fully automatic to generate comic books from videos sans human intervention, said the researchers in the paper. From an input video with subtitles, the computing algorithm first extracts informative key frames by analyzing the subtitles. It then stylizes the key frames into comic-style images.

Following this, the system uses a framework with a multi-page layout. This is to spread the images across a number of pages and produce visually appealing designs that reflect the connection between the images.

Unlike the speech balloons of most comic generation frameworks, the new system developed by researchers generate different types of balloons. It reflects emotions conveyed by words used by characters. To do this, it first tries to capture the emotion transferred by different lines of a dialog. It does so by analyzing a video audio track as well as the corresponding subtitles.