[FFmpeg-devel] GSoC mentored project: derain filter

[Liu Steven]

> 在 2019年2月20日，下午6:35，孟学苇 <xwmeng at pku.edu.cn> 写道：
> 
> Hi Dev-Community,
> 
> 
> 
> 
> I am Iris Meng from China. I’m a PhD student in Institute of Digital Media, Peking University. I wish to contribute as a GSoC applicant this year.
> 
> I am interested in Deep Learning. I want to add a derain filter in ffmpeg. If you have any suggestion or question, we can contact by email. My motivation and plans are as follows.
> 
> 
> 
> 
>       Motivation
> 
> Rain and fog are very common weather in actual life. However, it can affect the visibility. Especially in heavy rain, rain streaks from various directions accumulate and make the background scene misty, which will seriously influence the accuracy of many computer vision systems, including video surveillance, object detection and tracking in autonomous driving, etc. Therefore, it is an important task to remove the rain and fog, and recover the background from rain images. It can be used for image and video processing to make them clearer and it can be a preprocessing method for many computer vision systems.
> 
> 
> 
> 
>       Proposed Idea
> 
> We propose to implement this technology in ffmpeg. For video [1][2], we can utilize the relationship between frames to remove rain and fog. For single image [3], we can use traditional methods, such as discriminative sparse coding, low rank representation and the Gaussian mixture model. We can also use some deep learning methods. We should investigate these methods, and ultimately consider the effect of rain/fog removal and the complexity of the algorithm, and choose the optimal scheme.
> 
> 
> 
> 
>       Practical application
> 
> The derain and dehaze method can improve the subjective quality of videos and images.
> 
> 
> 
> 
>      Development plan
> 
> I would like to start working on my qualification task and try to solve my problems. Overall, I will follow the following steps to complete the project.
> 
> (1)    Literature and algorithm investigation
> 
> (2)    Data sets preparation
> 
> (3)    Coding: Implement network, training code, inference code and so on
> 
> (4)    Select the best method and transplantation it into ffmpeg
> 
> 
> 
> 
>      Reference
> 
>      [1] Zhang X, Li H, Qi Y, et al. Rain removal in video by combining temporal and chromatic properties[C]//2006 IEEE International Conference on Multimedia and Expo. IEEE, 2006: 461-464.
> 
>      [2] Tripathi A K, Mukhopadhyay S. Removal of rain from videos: a review[J]. Signal, Image and Video Processing, 2014, 8(8): 1421-1430.
> 
>      [3] Li X, Wu J, Lin Z, et al. Recurrent squeeze-and-excitation context aggregation net for single image deraining[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 254-269.
> 
> 
> 
I think this can reference libavflter/sr.c to implementation, maybe you can try two ways to implement it, one is native and the other is model.


Thanks
Steven

> 
> 
> 
> 
> Thanks,
> 
> Regards,
> 
> Iris Meng
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel