[FFmpeg-devel] Custom Video Encoding and Decoding

[Jaeho Bang]

Dear members of ffmpeg-devel,

I am currently working on a research project and from looking over the
examples and documentation in the ffmpeg repository, I believe I need to
implement a custom encoder / decoder to perform my task.

To first explain the project idea, currently I am working mainly with
surveillance footages. Using a traditional video decoder, I convert the
video into series of image frames and perform clustering to create frame
groups. Then using some performance metric, my algorithm selects a
representative frame from each group. By implementing a custom encoder and
decoder, I aim to improve the speed of extracting these representative
frames.

To further explain, I would like the custom encoder to be able to force the
frames I have chosen (representative frames) to be i-frames. The assumption
is that when analyzing videos, we only need to evaluate these
representative frames instead of the whole video (this I have already
experimentally confirmed), and extracting i-frames is faster than
extracting a combination of i-frames, b-frames, and p-frames.

Also, I would like to implement a decoder to quickly extract the series of
frames I give as input (they will all be i-frames). Through preliminary
experiments, I realized that extracting only i-frames by using the command
'select pict_type\I' query command only gives a 2x speed boost compared to
extracting all frames within the video. Hopefully, by implementing a custom
decoder, I aim to further increase this extraction speed.

Is there any code I can refer to? Is my assumption of 'if I were to
implement a custom decoder, the extraction speed would be faster than using
the keyword select to get all the i frames' valid?

Any help would be greatly appreciated. Thank you and have a wonderful day.

Sincerely,
Jaeho Bang

-- 
*Jaeho Bang*
*PhD Student in Computer Science*
Georgia Institute of Technology
North Ave NW, Atlanta, GA 30332, USA