[FFmpeg-devel] [PATCH] Added support for MB_INFO

[Carotti, Elias]

Hi all,
any chance someone could possibly have a look at this patch, please?
Thanks in advance
Elias


On Wed, 2023-05-03 at 12:27 +0000, Carotti, Elias wrote:
> Sorry to revive an old thread, but I updated the patch for ffmpeg 6
> and
> this new patch should address the comments.
> Still this is a libx264-only patch, and provides a means to specify
> that only portions of the frame have changed from the previous one
> while the others should be P_SKIP-ped.
> 
> More specifically, now the data is fully contained in the side data
> and
> the actual map of macroblocks which changed from the previous frame
> is
> created from the side information at encoding time, much like what is
> done for the region of interest.
> 
> best,
> Elias
> 
> 
> On Fri, 2022-06-17 at 17:10 +0200, Timo Rothenpieler wrote:
> > CAUTION: This email originated from outside of the organization. Do
> > not click links or open attachments unless you can confirm the
> > sender
> > and know the content is safe.
> > 
> > 
> > 
> > On 17.06.2022 12:59, Carotti, Elias wrote:
> > > > 
> > > > Yes, exactly. It relies on x264 to free it.
> > > 
> > > Not really. It can rely on x264 if you explicitly want that
> > > behavior.
> > > If you do not set a deallocator, it remains the caller
> > > responsibility.
> > > 
> > > > What happens if x264 is not involved, but some other encoder,
> > > > which
> > > > does
> > > > not check for that kind of side data?
> > > > 
> > > > How does the caller know that the data was consumed, and the
> > > > ownership
> > > > passed on?
> > > 
> > > Again, you don't have to pass the ownership, and in fact in my
> > > use
> > > case
> > > I do not pass it since I actually recycle and update the same
> > > buffer
> > > for subsequent frames. If you do intentionally pass the ownership
> > > you
> > > need to be aware of what you are doing. As I said, I see a leak
> > > only in
> > > case of a programming error.
> > > Maybe we could explicitly state it in the comment section in
> > > mb_info.h:
> > > if you set the deallocator you're relinquishing ownership of the
> > > buffer.
> > 
> > I'm not sure if that's something you can sensibly do with side
> > data?
> > What if it gets buffered, copied, and so on?
> > 
> > > Plus, there's one more flag (b_mb_info_update) in libx264 which
> > > allows
> > > to pass back information to the caller about which macroblocks
> > > among
> > > the flagged ones were actually encoded as P_SKIP. I did not add
> > > support
> > > for that though but in the future somebody may want to.
> > 
> > Yes, it's very x264 specific.
> > But the side data is generic. If for some reason x264 does not
> > process a
> > frame, or any other encoder ends up getting used, the data will
> > leak
> > if
> > it relied on x264 to free it.
> > 
> > > In principle I could've put the buffer into the side information 
> > > and
> > > not just pass a pointer to it but I thought that it would require
> > > adding more semantics which would imply a stronger dependency on
> > > libx264.
> > > Right now, mb_info is a vector of uint8_t flags and the only
> > > possible
> > > value - to date - is X264_MBINFO_CONSTANT. What if, say, one day
> > > libx264 decides the buffer has to be a vector of uint16_t or
> > > still
> > > uint8_t but the flags are packed? On the other hand, this, AFAIK,
> > > is
> > > only supported by libx264. Other codecs may want to choose a
> > > different
> > > semantic for a similar field and the only possibility to make it
> > > generic is letting the caller handling the low level details.
> > 
> > I'm not aware of any other side data with a similar semantic. And
> > I'm
> > really not sure if it's sane or even valid to do it like that.
> > Can't the data be entirely contained within the side data?
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel at ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> > 
> > To unsubscribe, visit link above, or email
> > ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
> 
> 
> 
> 
> NICE SRL, viale Monte Grappa 3/5, 20124 Milano, Italia, Registro
> delle Imprese di Milano Monza Brianza Lodi REA n. 2096882, Capitale
> Sociale: 10.329,14 EUR i.v., Cod. Fisc. e P.IVA 01133050052, Societa
> con Socio Unico
> 
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".




NICE SRL, viale Monte Grappa 3/5, 20124 Milano, Italia, Registro delle Imprese di Milano Monza Brianza Lodi REA n. 2096882, Capitale Sociale: 10.329,14 EUR i.v., Cod. Fisc. e P.IVA 01133050052, Societa con Socio Unico