[FFmpeg-devel] [PATCH 0/2] add FFDIFFSIGN for comparators

[Ganesh Ajjanagadde]

This patch series draws upon remarks of Mark and Ronald regarding the lack
of safety of the common idiom return a - b for qsort comparators.

I made an observation that the interesting (x > y) - (x < y) idiom works well:
it not only avoids branches commonly not optimized by compilers when the ternary
operator method is used, but is also safe with respect to overflow. See e.g
the link:
https://stackoverflow.com/questions/10996418/efficient-integer-compare-function/10997428#10997428

On a suggestion from Nicolas, slightly modified by me, 1/2 adds FFDIFFSIGN macro to avutil/common, and
documents it and its rationale. Note that is quite generic, and in particular
works for all of the standard built in integral and numerical types. NaN's will
compare equal with everything, which seems reasonable and matches C++'s std::sort
AFAIK.

2/2 utilizes FFDIFFSIGN in comparators across the codebase. A few were left out
due to reasons mentioned in a comment for 2/2.

Ganesh Ajjanagadde (2):
  avutil/common: add FFDIFFSIGN macro
  all: use FFDIFFSIGN to resolve possible undefined behavior in
    comparators

 cmdutils.c                      |  2 +-
 cmdutils_opencl.c               |  2 +-
 ffmpeg.c                        |  3 +--
 libavcodec/aacsbr_template.c    |  2 +-
 libavcodec/motion_est.c         |  2 +-
 libavfilter/f_sendcmd.c         |  8 +++-----
 libavfilter/vf_deshake.c        |  3 +--
 libavfilter/vf_palettegen.c     |  2 +-
 libavfilter/vf_removegrain.c    |  5 +----
 libavformat/subtitles.c         |  9 +++------
 libavutil/common.h              | 11 +++++++++++
 libswresample/swresample-test.c |  2 +-
 12 files changed, 26 insertions(+), 25 deletions(-)

-- 
2.6.2