[FFmpeg-devel] [PATCH] avutil/log: in colored_fputs, convert string from utf8 to CP_ACP charset on Windows

[周泽华]

Thanks for your reply.

Let me first show my situation. I have a mp3 file, in which the metadata is in
CP936 charset (CP_ACP is system dependent and is CP936 on my Windows). I issue
'ffprobe path_to_file' to see the information of the file. In the
'prepare_app_arguments' function of cmdutils.c , the arguments of ffprobe are
converted to utf8, so 'path_to_file' is stored in utf8 in ffmpeg, while the
metadata is stored in CP936. Both 'path_to_file' and metadata are sent to log
via 'colored_fputs'. So we can only detect the charset of input string at
runtime, not build time. I searched the internet and did not find any better
way to detect the charset.

I think the commit is an improvement. Before the commit, 'path_to_file' is
always garbled. If the metadata of a mp3 file is in utf8 charset, the log
output is also garbled. After the commit, the log output is garbled when
string in CP_ACP is converted 'from utf8 to CP_ACP successfully', which
is wrong and happens rarely as I observe.

About the unneeded code: I thought setting pointers to original value or null
is good. Anyway, a patch without these unneeded code is attached.

2014-02-20 10:41 GMT+08:00 Michael Niedermayer <michaelni at gmx.at>:
> On Wed, Feb 19, 2014 at 04:09:48PM +0800, ZHOU Zehua wrote:
>> From: ZHOU Zehua <zhouzehua at gmail.com>
>>
>> The strings in ffmpeg are mostly in utf8 charset, but the default
>> charset of Windows Console is not utf8. As a result, the log output is
>> garbled. This commit adds code to convert string from utf8 to CP_ACP
>> before printed on Windows.
>>
>> Note: If the input string is not in utf8, the conversion is supposed to
>> fail and the original input string will be printed.
>> ---
>>  libavutil/log.c |   37 +++++++++++++++++++++++++++++++++++++
>>  1 file changed, 37 insertions(+)
>>
>> diff --git a/libavutil/log.c b/libavutil/log.c
>> index 38ce1e8..0389a1e 100644
>> --- a/libavutil/log.c
>> +++ b/libavutil/log.c
>> @@ -50,6 +50,10 @@ static pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
>>  static int av_log_level = AV_LOG_INFO;
>>  static int flags;
>>
>> +#if defined(_WIN32) && !defined(__MINGW32CE__)
>> +#include<windows.h>
>> +#endif
>
> however exactly this is supposed to be detected, it should probably be
> done in configure and set some appropriate define like maybe
> HAVE_CP_ACP or something
> maybe theres also a cleaner way to detect this i dont know
>
>
>
>> +
>>  #if HAVE_SETCONSOLETEXTATTRIBUTE
>>  #include <windows.h>
>>  static const uint8_t color[16 + AV_CLASS_CATEGORY_NB] = {
>> @@ -103,9 +107,35 @@ static int use_color = -1;
>>
>>  static void colored_fputs(int level, const char *str)
>>  {
>> +#if defined(_WIN32) && !defined(__MINGW32CE__)
>> +    int num_wchars, num_chars;
>> +    wchar_t *wstr = NULL;
>> +    const char *str_orig = NULL;
>> +    char *str_acp = NULL; /* str in CP_ACP charset */
>> +#endif
>> +
>>      if (!*str)
>>          return;
>>
>> +#if defined(_WIN32) && !defined(__MINGW32CE__)
>> +    /* convert UTF-8 to wide chars, and then convert wide chars to CP_ACP */
>> +    num_wchars = MultiByteToWideChar(CP_UTF8, MB_ERR_INVALID_CHARS, str, -1, NULL, 0);
>> +    if (num_wchars > 0) {
>> +        wstr = av_mallocz(sizeof(wchar_t) * num_wchars);
>> +        if (wstr) {
>> +            MultiByteToWideChar(CP_UTF8, 0, str, -1, wstr, num_wchars);
>> +            num_chars = WideCharToMultiByte(CP_ACP, 0, wstr, -1, NULL, 0, NULL ,NULL);
>> +            str_acp = av_mallocz(sizeof(char) * num_chars);
>> +            if (str_acp) {
>> +                WideCharToMultiByte(CP_ACP, 0, wstr, -1, str_acp, num_chars, NULL, NULL);
>> +                str_orig = str;
>> +                str = str_acp;
>> +            }
>> +            av_freep(&wstr);
>> +        }
>> +    }
>> +#endif
>> +
>>      if (use_color < 0) {
>>  #if HAVE_SETCONSOLETEXTATTRIBUTE
>>          CONSOLE_SCREEN_BUFFER_INFO con_info;
>> @@ -152,6 +182,13 @@ static void colored_fputs(int level, const char *str)
>>          fputs(str, stderr);
>>  #endif
>>
>
>> +#if defined(_WIN32) && !defined(__MINGW32CE__)
>> +    if (str_acp) {
>> +        str = str_orig;
>> +        str_orig = NULL;
>
> these look unneeded
>
>
>> +        av_freep(&str_acp);
>> +    }
>> +#endif
>
>
>
>
>>  }
>>
>>  const char *av_default_item_name(void *ptr)
>> --
>> 1.7.9.5
>>
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel at ffmpeg.org
>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>
> --
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> Breaking DRM is a little like attempting to break through a door even
> though the window is wide open and the only thing in the house is a bunch
> of things you dont want and which you would get tomorrow for free anyway
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-avutil-log-in-colored_fputs-convert-string-from-utf8.patch
Type: application/octet-stream
Size: 2718 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20140220/da8056ad/attachment.obj>