[FFmpeg-devel] possible farewell; dev policy clarification

[Ganesh Ajjanagadde]

On Sat, Oct 31, 2015 at 1:01 AM, Rostislav Pehlivanov
<atomnuker at gmail.com> wrote:
> I agree with the others. There's quite a lot of noise on the ML and having
> some of
> your trivial patches (e.g. replacing functions with FFmpeg defined ones)
> merged into
> one would help to decrease it.

Could be, but please then answer my question regarding my puzzlement
over how to send patches: both en-masse and single things have been
frowned upon. I am sending a patch set today with a different approach
that I hope people like better.

>
>>Currently I am trying to get rid of a
>>lot of the last aspect to create a foundation for myself to work on
>>other things.
> Huh? What's so unclean about the current codebase that you simply can't
> work on
> anything else? Can you really not sleep at night if you know there's an
> unchecked malloc
> call (never a problem with a non-ancient OS) somewhere in an obscure 90's
> video game
> decoder which theoretically works but people lost the only available
> samples 5 years ago?

Not all the stuff I deal with are such obscure codecs. But yes,
generally I am quite picky about these things. I suspect it comes from
the fact that I am more theoretical at heart in the work I do. I hence
strive towards perfection regardless of the possible impact of a
particular file. As Nicolas pointed out, focusing on technical purity
and ignoring some of the negative social aspects of e.g one-liners is
part of what led to the present juncture. That, together with a lack
of tact on my end regarding rate-limiting/smoothing of bursty patch
traffic.

> What other things? Also be careful, the definition of a "foundation to work
> on" could vary
> between people. Hence a reason for reviews.

The foundation of course may differ, and I want something that works
with all. There is some misunderstanding on e.g IRC that "I don't like
reviews". I very much do appreciate reviews. Perhaps I am more
vociferous about the code I write, but that is because when I write
such things, I am honestly convinced of their technical merit. I of
course do revise if a suitable point is raised, and have
dropped/amended patches many times in the past.

Anyway, I have thought about this stuff, and am exploring some more
careful ways of sending stuff to the ML that may be liked more.
Feedback is welcome when I send such things.

>
>>Part of that defense (for me) is examining compiler warnings,
> Many compilers (don't) warn about things that other compilers would. There
> compilers
> have versions which (don't) warn about some things that other versions do.
> It's pointless
> and conforming to a method that most compilers don't warn about can be
> hacky and may
> cause some compilers difficulty in actually compiling. It's best to choose
> the absolute
> least hacky way which works well with the newest (if sane, if not, newest
> sane) version (!) of
> the most popular compiler and doesn't emit a warning with the current "-W*"
> arguments or
> just keep the current status quo.

I completely agree with this. I have hence dropped "hacky" warning
fixes, and have toned down on that front, e.g I have dropped the
en-masse av_warn_unused_result stuff. However, I believe (and still do
believe) some do have benefit. In fact, warnings from obscure
compilers are sometimes useful because they might have engineered a
particular warning that others have not. For instance, MSVC has pretty
terrible warnings, but I found the weird abort() call leading to a
Michael's solution because of it. Wparentheses is mostly complete
noise, but I found an almost surely incorrect usage that is now being
asked for a review by the maintainer.

I will amend the style with which I send patches regarding warnings, though.
By the way, I also focus on the latest toolchains for essentially your
reasons, something I have commented upon in the past before.

>
>>fixing undefined behavior,
> How does one fixed something undefined? Anyway, absolutely nothing wrong
> with justified
> undefined behavior, it's just working around the fact that:
>>conforming to standards when possible
> the standards are not that well defined or policed or sometimes sane (not
> really the case with C) to
> begin with. Rust wants to change that but it just takes away the fun of
> programming (in C & others).

I don't understand this: standards are the base of portability. Sane
or insane, it does not matter - many of the things need to be
respected. Multimedia is in my view one of the places for seeing the
worst of software, in Jean-Baptiste Kempf's words "Everything is
broken". Some can be ignored though, and (for instance) I have not
sent patches for the 4096 character string literal stuff as I know it
is utterly pointless.
Comments from especially Michael, Nicolas, and wm4 have given me a
better sense for what things can matter and what things are mostly
non-issues.

> Any compiler which does something different than what other compilers do
> under
> undefined behavior which you rely on is in fact a not-very-sane compiler
> and makes people
> sad because if many people are stuck with that not-very-sane compiler it
> means there is a
> need for a special workaround for the not-very-sane compiler. Which then
> encourages other
> compilers to also have some other special way of doing undefined behavior and
> thus bring more
> ugly hacks. So yeah, sticking to the standard when possible is the best.

This point has been raised before. This is why some undefined behavior
fixing is not that important. Nevertheless, it is important enough for
e.g Google to have its fuzzers report undefined behavior as they find
them, and for some FATE clients to run ubsan (or other sanitizer)
builds.

>
> From another email in this thread:
>
>>None of these things are satisfying, but to be honest, I am not
>>looking for satisfaction from FFmpeg.
> So you're not finding your work satisfying, you're probably not getting
> paid, you're bothering
> yourself over getting other people bothered by your work, yet you still do
> it? This is beyond logic.
> I suggest you make your mind up and either find joy from doing something or
> money or both.
>
>>I am looking for a way to channel frittered away energy obtained
>>during my transition from undergraduate to graduate life and its
>>associated increase in flexibility of time allocation.
>>In other words, I want a way to spend my down-time that is technical
>>in character.
> You wrote this as if writing a job application; this is a mailing list of
> equal adults (more or less)
> interested in drama, popcorn and the development of FFmpeg, in that order :)

I tend to be formal with things, because this is where I come from.
I don't think it is an issue, but it is perhaps funny to some here :).

> Though if you wrote this in a job application I'd hire you. Which is again
> more or less what anyone
> would do after they graduate: finding a job. Though nowadays there are
> other alternatives becoming
> popular: "finding yourself (by exploring the world)", "football!",
> "NEETdom", "soccer", "FOOTBALL!",
> "professional Quake 3 player", etc.
>
>>av_gcd patches
>>algorithmic improvements
> Yes, your work on ff_ctz was very nice, kudos, I liked it.

Thanks.

>
> On 30 October 2015 at 11:34, Ganesh Ajjanagadde <gajjanag at mit.edu> wrote:
>
>> Hi all,
>>
>> Apologies for the length of the email, but there is a lot of stuff I
>> would like to cover due to my absence on IRC or other venues for "non
>> patch related" discussion.
>>
>> There has been a lot said and still being said regarding some patches
>> I sent and associated discussion. Issues not related to patches in a
>> technical sense are being discussed privately.
>>
>> In the meantime, I have noticed an increased level of hostility on
>> ffmpeg-devel lately, both from my end as well as from others. I do not
>> wish to escalate it further, and want to let work progress as smoothly
>> as possible.
>>
>> In fact, I repeat a statement I made earlier: if overall people think
>> that if I leave better and more work will be done for the project, I
>> will gladly do so. That is the only thing I wish for: FFmpeg becoming
>> better. To clarify a possible confusion on IRC: I by no means intend
>> to work on Libav - either I work for FFmpeg, or not work at all on
>> multimedia. This statement holds true regardless of what people
>> express about the work I do here, whether I am kicked out, etc.
>>
>> The reason I repeat this question of leaving is because I dug through
>> IRC archives, trying to understand better what people here think of
>> the work I do in light of the increasing hostility on the ML. I got
>> the sense that a lot of information that I could have received did not
>> reach me as I am, and will not be on IRC. I also got the sense that
>> there is a section of FFmpeg developers who are deeply against the
>> kinds of things I do and the way I handle them, hence prompting the
>> question.
>>
>> Assuming that you feel I give a net positive benefit to the project,
>> please read on.
>>
>> I am quite puzzled with some of the sentiments expressed there. I
>> illustrate below what puzzles me, and ask what I can do on my end:
>>
>> 1. "Sloppy" patches - I tend to give more verbose commit messages to
>> explain rationale than many here. I also outlined why I did not post
>> benchmarks initially. I myself still think they are unnecessary and
>> not useful for many of my proposed changes, but the community here
>> seemed to think otherwise. I have hence started posting benchmarks.
>> I generally try to give evidence for things that are more "universal"
>> in character when possible, so as not to skew towards any particular
>> machine architecture, etc. I do not agree that the log patches were
>> "sloppy" - bit accuracy improvements for log10 over log should have
>> been obvious. Simplification of the code, a subjective aspect, should
>> have been quite a reasonable assumption. Do people here not think such
>> things are obvious?
>>
>> 2. "Increasing warnings" - There were only 2 instances AFAIK where I
>> made a mistake, one in apedec. Another was in a test build. Both were
>> very quickly resolved. These issues were tiny compared to the general
>> level of warning cleanups achieved. Both were reasonable mistakes that
>> missed the eyes of reviewers. Please do let me know when I do increase
>> such things, I try to test things the best I can on my end. I will
>> strive to be more cautious in the future by incorporating the lessons
>> learnt with these two failures.
>>
>> 3. Patch sending mechanism - I have tried both approaches, namely an
>> en-masse single patch, versus individual tiny patches. Both were
>> frowned upon: the en-masse as "not fond of mass changes", and the
>> single patches as "incredibly noisy". Regarding pinging, I pinged a
>> single patch in a set of similar patches with a phrase regarding the
>> fact that the ping extends to other such similar patches. This ping
>> was ignored. I push (following the dev policy) one change after
>> sufficient delay, and it was frowned upon. I did not know what to do,
>> so I subsequently pinged each of the patches. This resulted in a bunch
>> of comments on IRC. For reference, these are the av_warn_unused_result
>> patches for avutil. Any suggestions for the future?
>>
>> 4. av_warn_unused_result not being useful en-masse for all headers -
>> This was something remarked upon in IRC. I did not get this
>> information in the form of reviews. Please suggest which headers to
>> target first: I am currently just going one by one in a natural order
>> (so that I can keep track of which ones have been addressed). Of
>> course, if people think av_warn_unused_result is inherently bad, that
>> is a separate issue that reflects an even deeper gulf.
>>
>> 5. General communication failures - Clearly, a lot of important stuff
>> from IRC is not reaching the ML. I (and some others here) do not use
>> IRC, and are pretty firm about not using it. What can be done to
>> address the communication gap? I would have at least understood the
>> deep hostility to the work I do if I had seen IRC logs a few weeks
>> back. The ML got only the "effects" and little of the "causes".
>> Overall, my sense is that people let the negative reaction bottle up,
>> venting every now and then on IRC, since for whatever reason it is
>> more comfortable, although technically both are publicly recorded and
>> nearly as easily referenced in mails or commit messages. As a side
>> note, even some patch review is done on IRC, a situation I do not like
>> but have not raised so far as I am not affected by it.
>>
>> 6. General uselessness of the work I do - I agree a lot of work done
>> recently has been towards "theoretical" issues, etc. However, in the
>> past I have fixed tickets and cleaned up more "practical things" like
>> unchecked mallocs. Even now, I do address integer overflows, etc. I
>> have also done some performance optimization work at an algorithmic
>> level. I have had discussions with Michael privately regarding
>> optimizations. The reason I currently focus on "theoretical work" is
>> as follows:
>> a) Since it is scattered across the codebase, I get to see a variety
>> of files and API's and get a feel for things.
>> b) I feel that we have plenty of people who address new filters, new
>> asm optimizations, new demuxers, etc. I focus on work that others are
>> not willing or not interested in, so as to keep some balance. Quite
>> naturally people do not like these things as much and it generates
>> noise on the ML. I see no easy solution apart from avoiding such work,
>> which I think is a bad idea overall.
>> c) I viewed, and still view FFmpeg as a long term project for me to
>> work on. As such, I do not mind spending 6 months - 1 year cleaning up
>> the code in various ways.
>> d) I prefer to be defensive with respect to C code. Part of that
>> defense (for me) is examining compiler warnings, fixing undefined
>> behavior, conforming to standards when possible, etc. FATE should pass
>> before or after any of this, FFmpeg will continue to work fine
>> with/without this stuff. Why bother then? It is mainly for
>> sustainability and security reasons. The codebase continues to grow at
>> a rapid rate. I consider the above work a useful "line of defense"
>> against attack. By silencing warnings, new useful warnings/dangerous
>> practices can be spotted more readily. By fixing undefined behavior
>> etc assuring security of some components is easier. I thus think that
>> this is very different from being a "K & R style guy". In fact, very
>> few (or perhaps none) of my commits are reindents. There was the
>> avfilter rename work which people did express some interest in (see
>> Paul's commits) that I dropped quickly upon Stefano's remark. I did
>> this solely because no one else would do it, though some felt it could
>> be useful for consistency. In particular, I did not want "tribalism"
>> in libavilter - Paul's filters using one convention, Clement's
>> another, and so on and I expressed this reasoning publicly as well. If
>> people feel I am a "K & R style guy", I will leave as I don't think a
>> "K & R style guy" is useful for FFmpeg at the present juncture.
>> e) I have no personal interest in asm optimizations (to address a
>> suggestion on IRC). I do appreciate the hard work there, it just does
>> not appeal to me for reasons that I can answer privately. I am
>> interested in security aspects, algorithmic improvements, bug fixes,
>> and clean up (in that order). Currently I am trying to get rid of a
>> lot of the last aspect to create a foundation for myself to work on
>> other things. Part of that transition is already under way - see e.g
>> av_gcd patches as an algorithmic improvement.
>>
>> Please feel free to add things, send private email, or public
>> questions so that we can come to a better understanding.
>>
>> Regards,
>> Ganesh
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel at ffmpeg.org
>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel