[FFmpeg-devel] [PATCH] configure: speedup x2-x8

[avih]

Thanks.
 I'll post the 3 parts of main.patch unmodified as individualemails like Timo requested, and then I'll look at the failures.
 

    On Saturday, August 25, 2018 9:00 PM, Michael Niedermayer <michael at niedermayer.cc> wrote:
 

 On Sat, Aug 25, 2018 at 02:33:03PM +0000, avih wrote:
> Hi,
> 
> I noticed that configure can be a bit slow - few minutes on my macOS
> system, 1:30m on linux/bash, 30s on linux/dash, and even if
> pathological, still a too-long 10 minutes on Windows (MSYS2) (same
> duration as a full build with make -j4 after configure).
> 
> I added some timing information printouts for various parts in
> configure, and eventually identified few culprits.
> 
> The attachment "config-timing.patch" adds these printouts and sorted
> summary. It's attached for reference and not intended to be merged.
> It applies cleanly before or after the main patch.
> 
> 
> The attached "main.patch" addresses three areas I identified as slow:
> 
> 1. About 50-70% of configure runtime was being spent inside one
>    function: flatten_extralibs() and callees resolve() and unique().
>    It manipulates strings and invoked nearly 20K (20000) subshells.
>    It was rewritten to avoid subshells, and ended up x50-x250 faster.
> 
> 2. print_enabled_components() was invoking sed about 350 times on one
>    file. This is never instant but takes many seconds where fork is
>    slow (Windows). Invoke sed only once instead = x4-x10 speedup.
> 
> 3. After the previous speedups, configure spent 20-60% of its runtime
>    at check_deps(). It's particularly slow with bash. After some local
>    optimizations - mainly avoid pushvar/popvar and abort early in one
>    notable case (empty deps), it's now x4-x25 faster.
> 
> 
> Some resulting speedups (more details at "before-after.txt"):
> 
> - macOS/bash:  total: 98s -> 22s    flatten_extralibs:  53s -> 0.7s
> - Linux/bash:  total: 87s -> 11s    flatten_extralibs:  59s -> 0.9s
> - Linux/dash:  total: 27s ->  8s    flatten_extralibs:  17s -> 0.2s
> - FreeBSD/sh:  total: 34s ->  9s    flatten_extralibs:  23s -> 0.2s
> - MSYS2/bash:  total: 10m -> 2:30m  flatten_extralibs: 400s -> 1.5s (!)
> 
> 
> Notes:
> 
> - unique() is modified with flatten_extralibs. It now outputs different
>   order: it was keeping the last instance of recurring items, now it
>   keeps the first. It affects libs order at ffbuild/config.{mak,sh} -
>   but I don't think it matters. If it does, "opt1-reorder-unique.patch"
>   restores the original order. Let me know if/why it matters and I'll
>   squash it and update the commit message accordingly if required.
> 
> - After the check_deps() patch, pushvar() and popvar() are not used but
>   I was hesitant to remove them (nice to have). If you think it should
>   be removed, the patch "opt2-remove-pushvar.patch" removes them.
> 
> - The patches assume POSIX shell and don't use anything "tricky".
>   It was tested with dash, bash, busybox-ash, freebsd-sh, ksh93u, mksh.
> 
> - Thanks to tmm1 and atomnuker for their help with testing.
> 
> 
> This is my first mail to this list, and I wasn't sure how to split the
> patches. After asking a veteran, it was suggested to me that I should
> put all of them in one email. Just let me know if you prefer something
> else (dev mailing lists are not my forte...).

with the main patch
make distclean ; dash ./configure  --enable-gpl && make -j12 testprogs
fails:

LD    libavfilter/tests/filtfmts
libavformat/libavformat.a(utils.o): In function `av_apply_bitstream_filters':
ffmpeg/libavformat/utils.c:5577: undefined reference to `av_bitstream_filter_filter'
libavformat/libavformat.a(codec2.o): In function `codec2_read_header_common':
ffmpeg/libavformat/codec2.c:74: undefined reference to `avpriv_codec2_mode_bit_rate'
ffmpeg/libavformat/codec2.c:75: undefined reference to `avpriv_codec2_mode_frame_size'
ffmpeg/libavformat/codec2.c:76: undefined reference to `avpriv_codec2_mode_block_align'
ffmpeg/libavformat/codec2.c:74: undefined reference to `avpriv_codec2_mode_bit_rate'
ffmpeg/libavformat/codec2.c:75: undefined reference to `avpriv_codec2_mode_frame_size'
ffmpeg/libavformat/codec2.c:76: undefined reference to `avpriv_codec2_mode_block_align'
libavformat/libavformat.a(spdifdec.o): In function `spdif_get_offset_and_codec':
ffmpeg/libavformat/spdifdec.c:63: undefined reference to `av_adts_header_parse'
ffmpeg/libavformat/spdifdec.c:63: undefined reference to `av_adts_header_parse'
libavformat/libavformat.a(spdifenc.o): In function `spdif_header_aac':
ffmpeg/libavformat/spdifenc.c:357: undefined reference to `av_adts_header_parse'
collect2: error: ld returned 1 exit status
make: *** [libavfilter/tests/filtfmts] Error 1

[...]

-- 
Michael    GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Freedom in capitalist society always remains about the same as it was in
ancient Greek republics: Freedom for slave owners. -- Vladimir Lenin_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel at ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel