f



UTF-8 encoding

Hi,

Im working with xmprpc/client and the server Im working against
requires utf8 for my strings.

I found this article with google:
http://redhanded.hobix.com/inspect/futurismUnicodeInRuby.html

Is this for real? Is there no workaround for me to get my data to the
server with uft8 encoding?

Best,
martin

0
balony (2)
8/11/2005 6:09:55 PM
comp.lang.ruby 48886 articles. 0 followers. Post Follow

6 Replies
1262 Views

Similar Articles

[PageSpeed] 11

balony@gmail.com wrote:
>
> Im working with xmprpc/client and the server Im working against
> requires utf8 for my strings.
>
> I found this article with google:
> http://redhanded.hobix.com/inspect/futurismUnicodeInRuby.html
>
> Is this for real? Is there no workaround for me to get my data to the
> server with uft8 encoding?

You can use iconv or you can run an external pipe through GNU recode.

utf8_string = Iconv.new('iso-8859-1', 'utf-8').iconv(iso_8859_string)


0
dave1312 (540)
8/11/2005 9:01:08 PM
Dave Burt skrev:
> balony@gmail.com wrote:
> >
> > Im working with xmprpc/client and the server Im working against
> > requires utf8 for my strings.
> >
> > I found this article with google:
> > http://redhanded.hobix.com/inspect/futurismUnicodeInRuby.html
> >
> > Is this for real? Is there no workaround for me to get my data to the
> > server with uft8 encoding?
>
> You can use iconv or you can run an external pipe through GNU recode.
>
> utf8_string = Iconv.new('iso-8859-1', 'utf-8').iconv(iso_8859_string)

I'm running on a windows system, that could be a problem?
Also i'm getting "test.rb:4: uninitialized constant Iconv (NameError)"
when testing.

-- 
Martin

0
balony (2)
8/11/2005 9:27:47 PM
* balony@gmail.com <balony@gmail.com>:
> 
> Dave Burt skrev:
>> balony@gmail.com wrote:
[...]
>> > Is this for real? Is there no workaround for me to get my data to the
>> > server with uft8 encoding?
>>
>> You can use iconv or you can run an external pipe through GNU recode.
>>
>> utf8_string = Iconv.new('iso-8859-1', 'utf-8').iconv(iso_8859_string)
> 
> I'm running on a windows system, that could be a problem?  Also
> i'm getting "test.rb:4: uninitialized constant Iconv (NameError)"
> when testing.

You'll need to put the line:

require 'iconv'

at the top of your program to use the Iconv.* functions. I don't
have access to a Windows box, but if it's part of Ruby's standard
library, I'd assume it's available on Windows.

    - jeff -

-- 
jeff mital
jmital@aracnet.com 
Unix is simple. It just takes a genius to understand its simplicity.
        - Dennis Ritchie
0
jmital (1)
8/11/2005 11:58:12 PM
>>> utf8_string = Iconv.new('iso-8859-1', 'utf-8').iconv(iso_8859_string)
>
>> I'm running on a windows system, that could be a problem?  Also
>> i'm getting "test.rb:4: uninitialized constant Iconv (NameError)"
>> when testing.
>
>
>You'll need to put the line:
>
>require 'iconv'
>
>at the top of your program to use the Iconv.* functions. I don't
>have access to a Windows box, but if it's part of Ruby's standard
>library, I'd assume it's available on Windows.

The One-Click Installer still doesn't include Iconv (the developers are
tracking it as an open issue).

The Rails wiki has some info on how to get it going:
http://wiki.rubyonrails.com/rails/show/iconv
(See the Windows section.)
To summarize, the Ruby-mswin32 distribution at
http://www.garbagecollect.jp/ruby/mswin32/en/download/release.html
contains the required files.
You can also get the required iconv.dll from gettext on SourceForge:
http://sourceforge.net/project/showfiles.php?group_id=25167&package_id=51458

Cheers,
Dave

0
dave.burt (43)
8/12/2005 12:55:40 AM
Hi,
----- Original Message ----- 
From: <balony@gmail.com>
Newsgroups: comp.lang.ruby
To: "ruby-talk ML" <ruby-talk@ruby-lang.org>
Sent: Friday, August 12, 2005 6:31 AM
Subject: Re: UTF-8 encoding


>
>
> Dave Burt skrev:
>> balony@gmail.com wrote:
>> >
>> > Im working with xmprpc/client and the server Im working against
>> > requires utf8 for my strings.
>> >
>> > I found this article with google:
>> > http://redhanded.hobix.com/inspect/futurismUnicodeInRuby.html
>> >
>> > Is this for real? Is there no workaround for me to get my data to the
>> > server with uft8 encoding?
>>
>> You can use iconv or you can run an external pipe through GNU recode.
>>
>> utf8_string = Iconv.new('iso-8859-1', 'utf-8').iconv(iso_8859_string)
>
> I'm running on a windows system, that could be a problem?
> Also i'm getting "test.rb:4: uninitialized constant Iconv (NameError)"
> when testing.
>
> --
> Martin
>
Try this:

      def encode(str)
        begin
            require 'Win32API'
            str += "\0"
            ostr = "\0" * 256
            multiByteToWideChar = 
Win32API.new('kernel32','MultiByteToWideChar',['L','L','P','L','P','L'],'L')
            multiByteToWideChar.Call(0,0,str,-1,ostr,128)
            (ostr.strip + "\0").unpack("S*").pack("U*")
        rescue LoadError
            require 'iconv'
            Iconv::iconv('iso-8859-1','UTF-8',str)[0]
        end
      end

 utf8_string = encode(iso_8859_string)

Regards,

Park Heesob








0
phasis (35)
8/12/2005 1:14:24 AM
Hi,

Am Freitag, 12. Aug 2005, 03:11:11 +0900 schrieb balony@gmail.com:
> 
> Is this for real? Is there no workaround for me to get my data to the
> server with uft8 encoding?

I convert utf16 to utf8 using the workaround below. Having a
short look at Wikipedia, e.g., should lead to a similar
solution easily.

Bertram


------------------------------


def utf16to8 utf16
  endian = if utf16.slice! /\A\xff\xfe/ then
    "v"
  elsif       utf16.slice! /\A\xfe\xff/ then
    "n"
  else
    "S"
  end
  utf8 = ""
  utf16.scan /../ do |bb|
    b, = bb.unpack endian
    if b < 0x80 then
      utf8 << b
    elsif b < 0x800 then
      utf8 << (0b11000000 | (b >>  6 &  0b11111))
      utf8 << (0b10000000 | (b       & 0b111111))
    elsif b < 0x10000 then
      utf8 << (0b11100000 | (b >> 12 &   0b1111))
      utf8 << (0b10000000 | (b >>  6 & 0b111111))
      utf8 << (0b10000000 | (b       & 0b111111))
    else
      utf8 << (0b11110000 | (b >> 18 &    0b111))
      utf8 << (0b10000000 | (b >> 12 & 0b111111))
      utf8 << (0b10000000 | (b >>  6 & 0b111111))
      utf8 << (0b10000000 | (b       & 0b111111))
    end
  end
  utf8
end


-- 
Bertram Scharpf
Stuttgart, Deutschland/Germany
http://www.bertram-scharpf.de


0
lists2051 (431)
8/13/2005 9:19:44 AM
Reply: