[FFmpeg-devel] [PATCH] avformat/tls_openssl: fix build error when openssl version < 3

[Martin Storsjö]

On Thu, 5 Jun 2025, Jack Lau wrote:

>> On Jun 5, 2025, at 15:02, Martin Storsjö <martin at martin.st> wrote:
>> 
>> On Thu, 5 Jun 2025, Jack Lau via ffmpeg-devel wrote:
>> 
>>> fix the missing data structure pkey in the tls_context
>>> 
>>> Signed-off-by: Jack Lau <jacklau1222 at qq.com>
>>> ---
>>> libavformat/tls_openssl.c | 30 +++++++++++++++++-------------
>>> 1 file changed, 17 insertions(+), 13 deletions(-)
>> 
>> Thanks, this does fix the build break. However, I don't quite understand the fix...
>> 
>>> diff --git a/libavformat/tls_openssl.c b/libavformat/tls_openssl.c
>>> index b589d5d90a..bddeee9af8 100644
>>> --- a/libavformat/tls_openssl.c
>>> +++ b/libavformat/tls_openssl.c
>>> @@ -467,6 +467,7 @@ typedef struct TLSContext {
>>>    TLSShared tls_shared;
>>>    SSL_CTX *ctx;
>>>    SSL *ssl;
>>> +    EVP_PKEY *pkey;
>>> #if OPENSSL_VERSION_NUMBER >= 0x1010000fL
>>>    BIO_METHOD* url_bio_method;
>>> #endif
>> 
>> As far as I can see, nothing ever sets this new field, it is only used in a couple of places?
> Thanks for the review.
>
> The previous build error occurred because I forgot to properly set the 
> EC_KEY when using OpenSSL versions earlier than 3.0.
>
> In the current WHIP implementation, I initialize the key and certificate 
> (either by reading from file or generating them) before the DTLS 
> handshake, since the SDP requires fingerprints. The WHIP layer then 
> passes the key and certificate content as strings into the DTLS context.
>
> This fix ensures that the EVP_PKEY is loaded into the tls_context when 
> DTLS starts. For OpenSSL versions below 1.0.2, we need to call 
> SSL_CTX_set_tmp_ecdh, which requires an EC_KEY. So, i extract the EC_KEY 
> from the EVP_PKEY.
>
> I hope that explanation was clear—please feel free to reach out if you 
> have any further questions.

No that didn't answer my question.

As far as I can see, nothing sets the context variable p->pkey. It is used 
in openssl_init_ca_key_cert and later in dtls_start. But nothing ever sets 
p->key, so it will be NULL everywhere.

Did you test this code with openssl 1.0.2 (which those codepaths are for)?

It looks to me like this maybe should have an assignment in 
openssl_init_ca_key_cert, setting "p->key = pkey;" maybe?

// Martin