[FFmpeg-devel] [FFmpeg-cvslog] lavf/assenc: normalize line endings to \n

[Ridley Combs]

> On Feb 13, 2024, at 04:33, Martin Storsjö <martin at martin.st> wrote:
> 
> On Tue, 13 Feb 2024, Ridley Combs via ffmpeg-devel wrote:
> 
>>> On Feb 13, 2024, at 01:28, Anton Khirnov <anton at khirnov.net> wrote:
>>> Quoting Martin Storsjö (2024-02-12 12:31:29)
>>>> On Mon, 12 Feb 2024, Hendrik Leppkes wrote:
>>>>> On Mon, Feb 12, 2024 at 11:22 AM Martin Storsjö <martin at martin.st> wrote:
>>>>>>> diff --git a/.gitattributes b/.gitattributes
>>>>>>> index 5a19b963b6..a900528e47 100644
>>>>>>> --- a/.gitattributes
>>>>>>> +++ b/.gitattributes
>>>>>>> @@ -1,2 +1 @@
>>>>>>> *.pnm -diff -text
>>>>>>> -tests/ref/fate/sub-scc eol=crlf
>>>>>> This change seems to have had a tricky effect on the
>>>>>> tests/ref/fate/sub-scc file. Previously, when checked out, users got the
>>>>>> file with CRLF newlines. When updating to this git commit, or past it,
>>>>>> that file remains untouched, with CRLF still present, and the
>>>>>> fate-sub-scc test fails. If one does "rm tests/ref/fate/sub-scc; git
>>>>>> checkout tests/ref/fate/sub-scc", then the file does get restored with LR
>>>>>> newlines, and the test passes.
>>>>>> It's easy to do this change manually in the source checkout of a fate
>>>>>> runner, but I'm not sure how easily we get all fate instances fixed that
>>>>>> way - currently this test is failing in most of them.
>>>>> Can this be fixed by restoring the .gitattribute entry but with eol=lf?
>>>>> Not sure if Git would reset the file then.
>>>> No, that doesn't seem to make any difference. Not sure if there are any other straightforward/elegant fixes, short of renaming the file, which I guess would require renaming the test itself.
>>> I'm fine with renaming the test, unless anyone has a better fix.
>> 
>> We could probably tweak the fate runner script to make sure this gets fixed up; can anyone try this patch on one of the affected machines? https://gist.github.com/rcombs/c2ad470bf36c5cbd3fc33e699330eb15
> 
> That doesn't seem to make any difference.
> 
> Also, updating fate.sh doesn't necessarily propagate automatically to runners - in order to run fate, one needs to run fate.sh before it even clones/checks out the directory where it fetches the latest source. So unless one later has changed one's setup, to invoke a fate.sh from the checkout, most fate runners just use whatever copy of fate.sh they had when it was set up.
> 
>> Alternately, we could set -text on all fate ref files, or explicitly set eol=of for them, to ensure their line endings never get rewritten like this regardless of git config. I think either of these solutions would fix this in fate, but only after the fix commit gets checked out *followed by* at least one other commit.
> 
> Neither of those seem to make any difference either.
> 
> It's quite easy to test for one self:
> 
> $ git checkout -b experiment
> $ <commit change to .gitattributes>
> $ <commit another stray change if necessary>
> 
> $ git checkout 7bf1b9b3576~ # Reset original state, for testing
> $ rm tests/ref/fate/sub-scc; git checkout tests/ref/fate/sub-scc
> $ vi tests/ref/fate/sub-scc # inspect that the file originally has CRLF
> $ git checkout experiment~ # check out the commit setting attributes
> $ git checkout experiment # check out the next commit, with the new attributes set
> $ vi tests/ref/fate/sub-scc # observe that the file still has CRLF
> 
> $ git checkout --detach
> $ git -c core.autocrlf=false reset --hard 7bf1b9b3576
> $ vi tests/ref/fate/sub-scc # observe that the file still has CRLF

It looks like checkout has different behavior from reset, and fate uses a hard reset.
To test, I committed the change adding tests/ref/** -text, unix2dos'd tests/ref/fate/sub-scc, then ran git -c core.autocrlf=true reset --quiet --hard; this dos2unix'd the file as expected when run with a working tree containing the .gitattributes change (but not otherwise).

> 
> 
> It seems to me (I haven't trid to dig into manuals) that the attribute gets stuck in whatever form it was when the file was first created in the workdir. E.g. doing a "git checkout d1df72a702~" (the commit before the file was originally added) followed by "git checkout 7bf1b9b3576" does fix it. This is at least observed with git 2.25.1. Not sure if this is intended behaviour or a bug from git's side.

Git doesn't have any "memory" of the CRLFiness of a file beyond the content of the file itself (whether in the working tree or in committed blobs). It just doesn't necessarily replace every file in checkout invocations when they differ only in line endings. Windows was a mistake.

> 
> // Martin