[FFmpeg-devel] [PATCH 4/4] avfilter/vf_v360: refactor (i)flat_range for fisheye
Paul B Mahol
onemda at gmail.com
Sun Apr 11 02:37:09 EEST 2021
On Mon, Mar 22, 2021 at 1:35 PM Daniel Playfair Cal <
daniel.playfair.cal at gmail.com> wrote:
> > I disagree, if I use 180 hfov and 180 vfov it should not have extra
> areas but only half of previous input.
>
> Not sure I follow - the ih_fov and vh_fov refer to the input (i.e. the
> fisheye image). If you wanted to restrict the FoV of the output, surely the
> way to do that would be to implement and use the FoV settings for the
> equirectangular projection?. It doesn't seem right that the code for the
> input projection is responsible for deciding what appears in the output. My
> understanding was that the FoV settings simply describe the focal length of
> the input or output camera so that points in the images can me mapped
> to/from 3d coordinates.
>
> To give you an idea of what I am trying to fix, here is an example input:
> https://photos.app.goo.gl/o51NfY6aqWn3unPG6
> This is a 1920x1440 image taken on a GoPro Hero 5 black with the 4:3 Wide
> FoV setting and stabilisation disabled.
>
> The following filtergraph demonstrates the issues:
> 'v360=input=fisheye:ih_fov=116.66:iv_fov=87.50:output=flat:d_fov=145.8'
> 1. the dfov_from_hfov issue is worked around by the use of ih_fov and
> iv_fov instead of id_fov, although you can try with id_fov=145.8 to see
> that problem too
>
AFAIK, the h/v/d fov works fine with fisheye in/out. I used synthetic
fisheye images from paul bourke site.
And diagonal fov from w/h either works with both in and out or not at all.
> 2. by default the output has double the aspect ratio of the input, even
> though the fisheye -> rectilinear transformation doesn't change the aspect
> ratio (assuming the entire input image is included as it is in this example)
> 3. much of the input is not visible in the output even though there is a
> mapping between the chosen projections (changed in the visibility test
> patch)
>
> 3 in particular I don't think can be solved by changing the settings - the
> input field of view needs to match the FoV of the input camera, otherwise
> the mapping is wrong. But it seems there is no other way to include the
> entire input from a fisheye image.
>
So you want to not discard pixels out of circle defined by fov? That
generally does not make sense to me as that may not gonna have actual
pixels that belong to output.
Make sure that your files have correct projection, fisheye in v360 is
strict equidistant mapping, and may not be what your input is actually.
>
> On Mon, Mar 22, 2021 at 5:59 PM Paul B Mahol <onemda at gmail.com> wrote:
>
>>
>>
>> On Mon, Mar 22, 2021 at 5:09 AM Daniel Playfair Cal <
>> daniel.playfair.cal at gmail.com> wrote:
>>
>>> I've tried that filtergraph and a few other similar ones and I'm not
>>> sure what you mean - what exactly is the regression?
>>>
>>> I tried it on this image with an equirectangular projection:
>>> https://wiki.panotools.org/images/0/01/Big_ben_equirectangular.jpg
>>>
>>> The only difference I can see is that there are less unmapped areas in
>>> the output with the patches, because the final mapping from the output
>>> equirectangular image to the intermediate fisheye image no longer fails to
>>> map some areas which are present in the fisheye image. I would describe
>>> this as an improvement?
>>>
>>
>> I disagree, if I use 180 hfov and 180 vfov it should not have extra areas
>> but only half of previous input.
>>
>>
>>>
>>> On Mon, Mar 22, 2021 at 3:30 AM Paul B Mahol <onemda at gmail.com> wrote:
>>>
>>>> Sorry, but I cannot apply this set as is, It makes at least one serious
>>>> regression.
>>>>
>>>> For example try this filtergraph:
>>>>
>>>>
>>>> v360=input=e:output=fisheye:h_fov=180:v_fov=180,v360=input=fisheye:output=e:ih_fov=180:iv_fov=180
>>>>
>>>> On Sun, Mar 21, 2021 at 1:45 PM Daniel Playfair Cal <
>>>> daniel.playfair.cal at gmail.com> wrote:
>>>>
>>>>> This changes the iflat_range and flat_range values for the fisheye
>>>>> projection to match their meaning for the flat/rectilinear projection.
>>>>> That is, the range is between the two x or two y coordinates of the
>>>>> outermost points above/below or left/right of the center, in the
>>>>> flat/rectilinear projection.
>>>>>
>>>>> Signed-off-by: Daniel Playfair Cal <daniel.playfair.cal at gmail.com>
>>>>> ---
>>>>> libavfilter/vf_v360.c | 19 +++++++++----------
>>>>> 1 file changed, 9 insertions(+), 10 deletions(-)
>>>>>
>>>>> diff --git a/libavfilter/vf_v360.c b/libavfilter/vf_v360.c
>>>>> index 68bb2f7b0f..3158451963 100644
>>>>> --- a/libavfilter/vf_v360.c
>>>>> +++ b/libavfilter/vf_v360.c
>>>>> @@ -2807,9 +2807,8 @@ static int prepare_fisheye_out(AVFilterContext
>>>>> *ctx)
>>>>> {
>>>>> V360Context *s = ctx->priv;
>>>>>
>>>>> - s->flat_range[0] = s->h_fov / 180.f;
>>>>> - s->flat_range[1] = s->v_fov / 180.f;
>>>>> -
>>>>> + s->flat_range[0] = 0.5f * s->h_fov * M_PI / 180.f;
>>>>> + s->flat_range[1] = 0.5f * s->v_fov * M_PI / 180.f;
>>>>> return 0;
>>>>> }
>>>>>
>>>>> @@ -2827,8 +2826,8 @@ static int fisheye_to_xyz(const V360Context *s,
>>>>> int i, int j, int width, int height,
>>>>> float *vec)
>>>>> {
>>>>> - const float uf = s->flat_range[0] * ((2.f * i) / width - 1.f);
>>>>> - const float vf = s->flat_range[1] * ((2.f * j + 1.f) / height -
>>>>> 1.f);
>>>>> + const float uf = 2.f * s->flat_range[0] / M_PI * ((2.f * i) /
>>>>> width - 1.f);
>>>>> + const float vf = 2.f * s->flat_range[1] / M_PI * ((2.f * j + 1.f)
>>>>> / height - 1.f);
>>>>>
>>>>> const float phi = atan2f(vf, uf);
>>>>> const float theta = M_PI_2 * (1.f - hypotf(uf, vf));
>>>>> @@ -2858,8 +2857,8 @@ static int prepare_fisheye_in(AVFilterContext
>>>>> *ctx)
>>>>> {
>>>>> V360Context *s = ctx->priv;
>>>>>
>>>>> - s->iflat_range[0] = s->ih_fov / 180.f;
>>>>> - s->iflat_range[1] = s->iv_fov / 180.f;
>>>>> + s->iflat_range[0] = 0.5f * s->ih_fov * M_PI / 180.f;
>>>>> + s->iflat_range[1] = 0.5f * s->iv_fov * M_PI / 180.f;
>>>>>
>>>>> return 0;
>>>>> }
>>>>> @@ -2882,10 +2881,10 @@ static int xyz_to_fisheye(const V360Context *s,
>>>>> {
>>>>> const float h = hypotf(vec[0], vec[1]);
>>>>> const float lh = h > 0.f ? h : 1.f;
>>>>> - const float phi = atan2f(h, vec[2]) / M_PI;
>>>>> + const float phi = atan2f(h, vec[2]);
>>>>>
>>>>> - float uf = vec[0] / lh * phi / s->iflat_range[0];
>>>>> - float vf = vec[1] / lh * phi / s->iflat_range[1];
>>>>> + float uf = 0.5f * vec[0] / lh * phi / s->iflat_range[0];
>>>>> + float vf = 0.5f * vec[1] / lh * phi / s->iflat_range[1];
>>>>>
>>>>> const int visible = -0.5f < uf && uf < 0.5f && -0.5f < vf && vf <
>>>>> 0.5f;
>>>>> int ui, vi;
>>>>> --
>>>>> 2.31.0
>>>>>
>>>>> _______________________________________________
>>>>> ffmpeg-devel mailing list
>>>>> ffmpeg-devel at ffmpeg.org
>>>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>>>>
>>>>> To unsubscribe, visit link above, or email
>>>>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>>>>
>>>>
More information about the ffmpeg-devel
mailing list