[FFmpeg-devel] [FFmpeg-cvslog] avcodec/mjpegdec: support weird RGB subsampling with progressive

[Leo Izen]

On 4/12/23 11:35, Michael Niedermayer wrote:
> Hi
> 
> On Mon, Apr 10, 2023 at 08:56:19AM +0000, Leo Izen wrote:
>> ffmpeg | branch: master | Leo Izen <leo.izen at gmail.com> | Thu Mar 16 15:32:14 2023 -0400| [b18a9c29713abc3a1b081de3f320ab53a47120c6] | committer: Leo Izen
>>
>> avcodec/mjpegdec: support weird RGB subsampling with progressive
>>
>> This allows weird subsampling with progressive JPEGs to be decoded,
>> such as full-RG and only B subsampled.
> 
> This breaks multiple jpeg files
> 
> tickets/4045/1x2.jpg
> tickets/4045/2x1.jpg
> 
> [...]
> 

I believe this should fix it. Should these two jpegs be added as FATE 
tests to prevent an issue like this in the future?

diff --git a/libavcodec/mjpegdec.c b/libavcodec/mjpegdec.c
index 3fc18aac73..01537d4774 100644
--- a/libavcodec/mjpegdec.c
+++ b/libavcodec/mjpegdec.c
@@ -1698,6 +1698,9 @@ int ff_mjpeg_decode_sos(MJpegDecodeContext *s, 
const uint8_t *mb_bitmask,
          s->h_scount[i]  = s->h_count[index];
          s->v_scount[i]  = s->v_count[index];

+        if(nb_components == 3 && s->nb_components == 3 && 
s->avctx->pix_fmt == AV_PIX_FMT_GBRP)
+            index = (index+2)%3;
+
          s->comp_index[i] = index;

          s->dc_index[i] = get_bits(&s->gb, 4);
@@ -2721,7 +2724,7 @@ the_end:
          }
      }

-    if (s->avctx->pix_fmt == AV_PIX_FMT_GBRP) {
+    if (s->avctx->pix_fmt == AV_PIX_FMT_GBRP && s->progressive) {
          av_assert0(s->nb_components == 3);
          FFSWAP(uint8_t *, frame->data[0], frame->data[2]);
          FFSWAP(uint8_t *, frame->data[0], frame->data[1]);