[FFmpeg-devel] [PATCH 0/3 v2] avcodec/aacdec_template: improvements to 22.2 layout logic

[Jan Ekström]

Compared to v1:
* Left and right are being handled separately in case of assign_pair()'s
  2x SCE mode. (patch 1/3)
* Exact coding element is also checked instead of just coding element
  channel type in sniff_channel_order()'s 22.2 specific if statement.
  (patch 2/3)
* Last check in sniff_channel_order() for 22.2 is simplified to just the
  channel layout. The if already does enough checking and if the input
  ended up getting the full 22.2 layout, we are good to go. (patch 2/3)

The first two commits fix both of the fuzzing samples I have on hand.
One being from #8845, and another provided privately by Michael. Changes have
been tested both with clang 10's ASAN as well as standard valgrind.

The last commit is general debug logging that I found helpful during
adding support of 22.2 by logging the element order before/after reorder.
This one, if found non-interesting, can just be ignored. I decided to
post it as it was part of my verification.

Jan Ekström (3):
  avcodec/aacdec_template: keep tabs on layout in sniff_channel_order
  avcodec/aacdec_template: add more checks to make sure only 22.2 gets
    to 22.2
  avcodec/aacdec_template: log the element order before/after reordering

 libavcodec/aacdec_template.c | 129 ++++++++++++++++++++++++++++-------
 1 file changed, 106 insertions(+), 23 deletions(-)

-- 
2.26.2