Jump to content


encoding problem with emoticon


4 replies to this topic

#1 capella

    Advanced Member

  • Members
  • PipPipPip
  • 49 posts

Posted 26 March 2018 - 08:24 AM

I see here a problem that is related to emoticons.
A chat partner has used an emoticon in one message. (I copied the emoticon in the edit window by copy & paste, but once I submitted I saw only some questions marks. The forum software might also have an encoding problem?)

I was surprised to see that this message was not visible in the chat archive anymore. Another message with a different emoticon was not transferred to the archive too.

I digged a bit in the database and found that:

1. Indeed, the message has not found it's way into the archive.

2. The corresponding data in table 'lz_chat_post' has an incorrect encoding in the field 'text'. phpadmin is complaining about it and an export to csv also generates incorrect UTF-8 data.

3. The message also contained an umlaut, so there is not a general problem with the encoding.

Anything else I can do to find out more, so that you can fix this?

Cheers Jens

#2 capella

    Advanced Member

  • Members
  • PipPipPip
  • 49 posts

Posted 26 March 2018 - 08:38 AM

I just see that emoticons have a very hight uncode: U+1F600 and more.

Others also have problems with this:
http://www.textasdat...ing-of-utf-816/

#3 capella

    Advanced Member

  • Members
  • PipPipPip
  • 49 posts

Posted 26 March 2018 - 08:44 AM

and here:

https://www.drupal.o.../issues/1314214

In my installation the collation is set to utf8_bin and not utf8mb4. So maybe this is the cause of the problem.

#4 Patrick Keil

    Administrator

  • Administrators
  • 3616 posts
  • LocationSingen, Germany

Posted 26 March 2018 - 12:17 PM

Upgrading to utf8mb4 is already on our list.

http://roadmap.livezilla.net/

Cheers.

#5 capella

    Advanced Member

  • Members
  • PipPipPip
  • 49 posts

Posted 26 March 2018 - 12:42 PM

Thanks, good to know this.

May I suggest to implement a work around until the real solution has been implemented?

I would remove all illegal characters from the string before the data is transferred to the database. Diskussion about this can be found here:
https://stackoverflo...8-string-in-php

Doing this, you would loose some characters but the message itself would survive and show up in the archive. Right now the whole message is gone.





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users