[Solved] Server Daemon needs UTF-8 Support

[houkouonchi]

Does not record things correctly in the db for songs with foreign characters. Needs utf-8 support.

[KandaRainbowsoul]

It doesn't record Vocaloid Collaboration songs at all
Although...It should've recorded FLOWER TAIL (VCP 8) if it was just this yet it didn't...

[Angelo]

Can anyone please confirm that my commit a733fb05534a24dc5073ce886725a7c5ce92919f is working?

I don't have any songs with special UTF8 characters in them.

[houkouonchi]

Doesn't appear to have fixed anything. I suspect that the non UTF8 support is causing the weird color issue as well:


http://box.houkouonchi.jp/screen00240.jpg

http://box.houkouonchi.jp/smo_utf8.png

Also an old bug with 3.95 has but its head into things again. Maybe the code just reverted somewhere? Gonna try re-adding the 3.95 support patch.

[houkouonchi]

Ok so fixed the 3.95 bug using the same code as before IE:

if (User_Protocol == 2)
gsuCtr += 2;

Also played the song again on the live DB and the percentage is right

http://stepmaniaonline.net/index.php?pa ... &mid=14101

The good news is I don't think this song was recording in the DB at all before because I know I have played it before.

Also I know UTF8 works on the site if its in the db as I have manually made my name in utf8 as a test and it displayed correctly on the site as 方向音痴

[foxfire667]

Hasn't been fixed yet unfortunately.

I used a different pack, and was on SM 5 if it makes any difference. Figured two sources is better than one I guess:

[houkouonchi]

I think this is more than just the connection to sqlite (although that might have been needed as well). I think the strings themselves need to be encoded as utf-8 for example:

http://stackoverflow.com/questions/8707 ... ng-to-utf8

[Angelo]

Can anyone give me a download to a song with UTF-8 characters in them?

Also, Hou, can you please make a pull request on Github for that if it's not in the Git repo yet?

Thanks!

[houkouonchi]

[Izzy]

If the database is now using MySQL does that mean it has UTF-8 support? Or at least it is possible now?

[houkouonchi]

[houkouonchi]

[houkouonchi]

Having a problem of the client disconnecting when its sent stuff with utf8 (not sure why). Until that gets worked out the not recording of some songs is still borked. However I am currently using a lame PHP workaround to reformat the data from the db and search strings so that it displays correctly on the site.

For example you can do a search like:

http://www.stepmaniaonline.net/index.ph ... =%E3%82%89

which is song=ら

[Cube]

Instead of changing the server code, we just changed the site code to decode the strings and print them correctly.

Houkouonchi finished making the changes to the site, and I finished the javascript to so the rooms pages print out correctly now as well.

[houkouonchi]

Server now has utf8 support (daemon) and database is in utf8 native (atleast for songs, not for users). This should also fix the songs not being recorded in the db.

[Thumbsy]

Playing again today... songs with foreign characters still fail to record, I checked my profile and none of the songs I played that have UTF8 characters in the artist and/or title field are displayed.

[houkouonchi]

[Thumbsy]

[Cube]

[houkouonchi]