[FFmpeg-devel] [PATCH] http: make caching of redirect url optional

[Eran Kornblau]

> 
> -----Original Message-----
> From: Eran Kornblau 
> Sent: Thursday, 2 December 2021 9:16
> To: FFmpeg development discussions and patches <ffmpeg-devel at ffmpeg.org>
> Subject: RE: [FFmpeg-devel] [PATCH] http: make caching of redirect url optional
> 
> Another ping... resending with the original patch
> 
> Eran 
> 
> -----Original Message-----
> From: Eran Kornblau 
> Sent: Thursday, 25 November 2021 8:55
> To: FFmpeg development discussions and patches <ffmpeg-devel at ffmpeg.org>
> Subject: RE: [FFmpeg-devel] [PATCH] http: make caching of redirect url optional
> 
> > > > 
> > > > On Tue, 2 Nov 2021, Eran Kornblau wrote:
> > > > 
> > > > > Hi all,
> > > > >
> > > > > The attached patch makes the default behavior of caching HTTP redirects optional.
> > > > 
> > > > Is caching a redirected URL allowed per the HTTP spec? If not, then no caching should take place, or the caching should be optional, and not the default.
> > > > 
> > > From a quick search on this, it seems that browsers by default cache 301 redirects, while 302/307 are not cached by default.
> > > However, in both cases, the browser is expected to honor any Cache-Control/Expires headers.
> > > 
> > > Changing the implementation to imitate the behavior of a browser would be more complex, since currently there is only a single "slot" in the cache.
> > > For example, if we have a chain of redirects 307->301->200, and we want to cache the 301 but not the 307, we can't really do it.
> > > Solving this one would require a more complex data structure, and if we add expiration/TTL to the mix, it becomes even more complex...
> > > 
> > > Assuming we don't want to add a large chunk of code to handle this, the options I see are - 1. A simple boolean option - as proposed in this patch. We can argue about what the default should be :) but it felt safer to me to have the default behavior as it was before.
> > > 2. Cache only 301's directly linked to the original URL - e.g. if we have 301->301->307->301->200, we cache only the first 2 redirects.
> > > My concern here is that this would break/slow down applications that rely on the existing behavior of caching any redirect.
> > > 
> > 
> > Hi,
> > 
> > Looking for some guidance how to proceed with this...
> > 
> > On one side, we have the minimal alternative of adding an option to the http module (as in the patch I submitted), on the other side we have the alternative of fully imitating the behavior of a browser - respect cache-control/expires, and if these headers are missing, default according to status code (301/302/307).
> > We can also go with something in between - e.g., we can cache 301's that are returned directly from the original url, and avoid caching any 302/307, as well as 301's that come after a 302/307.
> > 
> 
> Ping... 
> 
> Any objection to merge the patch that adds an option to disable the caching of redirect?
> 
> Thanks!
> 
> Eran
> 
> > Thank you
> > 
> > Eran
> > 
> > > Any thoughts/suggestions are welcome,
> > > 
> > > Thank you,
> > > 
> > > Eran
> > > 
> > > > Thanks,
> > > > Marton
> > > >
> > >
> >
> 
Trying again... we're stuck on this one...

To recap - I want to somehow prevent ffmpeg from caching HTTP redirects. 
The patch I submitted implements it using a new option, but I'm open to other suggestions - doing it based on HTTP status code etc.
Just need some guidance on which direction to take here, so that some solution for this could be merged. 

Thank you,

Eran