f



UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0: invalid start byte

I just started to have this error without changing nothing

in my index.html(template) and metrites.py(which ipen the template)

[Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97] Original 
exception was:
[Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97] Traceback 
(most recent call last):
[Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97]   File 
"/home/nikos/public_html/cgi-bin/metrites.py", line 19, in <module>
[Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97]     host = 
socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or 'UnResolved'
[Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97] 
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0: 
invalid start byte
[Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97] Premature end 
of script headers: metrites.py


Why cant it decode the starting byte? what starting byte is that?
-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/4/2013 8:37:08 AM
comp.lang.python 77058 articles. 3 followers. Post Follow

65 Replies
1210 Views

Similar Articles

[PageSpeed] 35

Am 04.07.2013 10:37, schrieb Νίκος:
> I just started to have this error without changing nothing

Well, undo the nothing that you didn't change. ;)

> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0:
> invalid start byte
> [Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97] Premature end
> of script headers: metrites.py
>
> Why cant it decode the starting byte? what starting byte is that?

It's the 0xb6 but it's expecting the starting byte of a UTF-8 sequence. 
Please do some research on UTF-8, that should clear it up. You could 
also search for common causes of that error.

Uli


0
7/4/2013 9:50:15 AM
On 07/04/2013 04:37 AM, Νίκος wrote:
> I just started to have this error without changing nothing
>
> in my index.html(template) and metrites.py(which ipen the template)
>
> [Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97] Original
> exception was: [Thu Jul 04 11:35:14 2013] [error] [client
> 108.162.229.97] Traceback (most recent call last): [Thu Jul 04
> 11:35:14 2013] [error] [client 108.162.229.97]   File
> "/home/nikos/public_html/cgi-bin/metrites.py", line 19, in <module>
> [Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97]     host =
>  socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or
> 'UnResolved' [Thu Jul 04 11:35:14 2013] [error] [client
> 108.162.229.97] UnicodeDecodeError: 'utf-8' codec can't decode byte
> 0xb6 in position 0: invalid start byte [Thu Jul 04 11:35:14 2013]
> [error] [client 108.162.229.97] Premature end of script headers:
> metrites.py
>
>
> Why cant it decode the starting byte? what starting byte is that?

The error message means that somebody is trying to decode a byte string
into Unicode, and using the utf-8 codec for it.  Only certain sequences
are legal in utf-8, and the first byte of a character may not be 0xb6.
So it gives an error.  The question is where does this string come from.


Well, the message shows the source line from metrites.py:

host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or 'UnResolved'

So the most likely candidate is the string in the environment named
"REMOTE_ADDR"  Can you display that string?  it should look like

"11.24.32.4"

or some other valid IP address.


I'm assuming Python 2.7.  You should specify the python version when 
starting a new thread, as we (or at least I) cannot keep track of what 
version everyone's running.




-- 
DaveA

0
davea9629 (629)
7/4/2013 9:59:22 AM
Στις 4/7/2013 12:59 μμ, ο/η Dave Angel έγραψε:
> On 07/04/2013 04:37 AM, Νίκος wrote:
>> I just started to have this error without changing nothing
>>
>> in my index.html(template) and metrites.py(which ipen the template)
>>
>> [Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97] Original
>> exception was: [Thu Jul 04 11:35:14 2013] [error] [client
>> 108.162.229.97] Traceback (most recent call last): [Thu Jul 04
>> 11:35:14 2013] [error] [client 108.162.229.97]   File
>> "/home/nikos/public_html/cgi-bin/metrites.py", line 19, in <module>
>> [Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97]     host =
>>  socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or
>> 'UnResolved' [Thu Jul 04 11:35:14 2013] [error] [client
>> 108.162.229.97] UnicodeDecodeError: 'utf-8' codec can't decode byte
>> 0xb6 in position 0: invalid start byte [Thu Jul 04 11:35:14 2013]
>> [error] [client 108.162.229.97] Premature end of script headers:
>> metrites.py
>>
>>
>> Why cant it decode the starting byte? what starting byte is that?
>
> The error message means that somebody is trying to decode a byte string
> into Unicode, and using the utf-8 codec for it.  Only certain sequences
> are legal in utf-8, and the first byte of a character may not be 0xb6.
> So it gives an error.  The question is where does this string come from.
>
>
> Well, the message shows the source line from metrites.py:
>
> host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or 'UnResolved'
>
> So the most likely candidate is the string in the environment named
> "REMOTE_ADDR"  Can you display that string?  it should look like
>
> "11.24.32.4"
>
> or some other valid IP address.
>
>
> I'm assuming Python 2.7.  You should specify the python version when
> starting a new thread, as we (or at least I) cannot keep track of what
> version everyone's running.

Ima using Python v3.3.2 Dave

The host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or 
'UnResolved' should not have give an error since i explicityl tell it 
that if it cannot resolve dns the ip to hostname to set it as "unresolved"

The error appear ONLY when i CloudFlare superhost.gr

If i pause tthe domain form CloudFlare then my website loads properly.


-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/4/2013 10:03:55 AM
On 07/04/2013 06:03 AM, Νίκος wrote:
> Στις 4/7/2013 12:59 μμ, ο/η Dave Angel έγραψε:
>> On 07/04/2013 04:37 AM, Νίκος wrote:
>>> I just started to have this error without changing nothing
>>>
>>> in my index.html(template) and metrites.py(which ipen the template)
>>>
>>> [Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97] Original
>>> exception was: [Thu Jul 04 11:35:14 2013] [error] [client
>>> 108.162.229.97] Traceback (most recent call last): [Thu Jul 04
>>> 11:35:14 2013] [error] [client 108.162.229.97]   File
>>> "/home/nikos/public_html/cgi-bin/metrites.py", line 19, in <module>
>>> [Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97]     host =
>>>  socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or
>>> 'UnResolved' [Thu Jul 04 11:35:14 2013] [error] [client
>>> 108.162.229.97] UnicodeDecodeError: 'utf-8' codec can't decode byte
>>> 0xb6 in position 0: invalid start byte [Thu Jul 04 11:35:14 2013]
>>> [error] [client 108.162.229.97] Premature end of script headers:
>>> metrites.py
>>>
>>>
>>> Why cant it decode the starting byte? what starting byte is that?
>>
>> The error message means that somebody is trying to decode a byte string
>> into Unicode, and using the utf-8 codec for it.  Only certain sequences
>> are legal in utf-8, and the first byte of a character may not be 0xb6.
>> So it gives an error.  The question is where does this string come from.
>>
>>
>> Well, the message shows the source line from metrites.py:
>>
>> host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or
>> 'UnResolved'
>>
>> So the most likely candidate is the string in the environment named
>> "REMOTE_ADDR"  Can you display that string?  it should look like
>>
>> "11.24.32.4"
>>
>> or some other valid IP address.
>>
>>
>> I'm assuming Python 2.7.  You should specify the python version when
>> starting a new thread, as we (or at least I) cannot keep track of what
>> version everyone's running.
>
> Ima using Python v3.3.2 Dave
>
> The host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or
> 'UnResolved' should not have give an error since i explicityl tell it
> that if it cannot resolve dns the ip to hostname to set it as "unresolved"

That's not true.  The 'or' doesn't get executed until after the 
gethostbyaddr() call has returned.  So if something's wrong with that 
call, and it throws an exception, the 'or "unresolved"' won't help.

I don't know that this is the problem, and I'm not all familiar with 
these api's.  But I cannot see anything else that could go wrong there 
to give that particular exception.  Unless the hostname it's going to 
return is a byte string.

>
> The error appear ONLY when i CloudFlare superhost.gr
>
> If i pause tthe domain form CloudFlare then my website loads properly.
>
>

I don't really know what CloudFlare is, and have no idea what 'pausing 
the form' will do.  But since it has something to do with dns, perhaps 
it's returning an invalid host name, one that isn't encoded in utf-8.

-- 
DaveA

0
davea9629 (629)
7/4/2013 10:29:25 AM
Στις 4/7/2013 12:50 μμ, ο/η Ulrich Eckhardt έγραψε:
> Am 04.07.2013 10:37, schrieb Νίκος:
>> I just started to have this error without changing nothing
>
> Well, undo the nothing that you didn't change. ;)
>
>> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0:
>> invalid start byte
>> [Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97] Premature end
>> of script headers: metrites.py
>>
>> Why cant it decode the starting byte? what starting byte is that?
>
> It's the 0xb6 but it's expecting the starting byte of a UTF-8 sequence.
> Please do some research on UTF-8, that should clear it up. You could
> also search for common causes of that error.

So you are also suggesting that what gesthostbyaddr() returns is not 
utf-8 encoded too?

What character is 0xb6 anyways?

-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/4/2013 10:38:09 AM
On Thu, Jul 4, 2013 at 8:38 PM, =CD=DF=EA=EF=F2 <nikos@superhost.gr> wrote:
> So you are also suggesting that what gesthostbyaddr() returns is not utf-=
8
> encoded too?
>
> What character is 0xb6 anyways?

It isn't. It's a byte. Bytes are not characters.

http://www.joelonsoftware.com/articles/Unicode.html

ChrisA
0
rosuav (4324)
7/4/2013 10:54:30 AM
On 04/07/2013 11:38, Νίκος wrote:
> Στις 4/7/2013 12:50 μμ, ο/η Ulrich Eckhardt έγραψε:
>> Am 04.07.2013 10:37, schrieb Νίκος:
>>> I just started to have this error without changing nothing
>>
>> Well, undo the nothing that you didn't change. ;)
>>
>>> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0:
>>> invalid start byte
>>> [Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97] Premature end
>>> of script headers: metrites.py
>>>
>>> Why cant it decode the starting byte? what starting byte is that?
>>
>> It's the 0xb6 but it's expecting the starting byte of a UTF-8 sequence.
>> Please do some research on UTF-8, that should clear it up. You could
>> also search for common causes of that error.
>
> So you are also suggesting that what gesthostbyaddr() returns is not
> utf-8 encoded too?
>
> What character is 0xb6 anyways?
>
Well, it's from a bytestring, so you'll have to specify what encoding
you're using! (It clearly isn't UTF-8.)

If it's ISO-8859-7 (what you've previously referred to as "greek-iso"),
then:

 >>> import unicodedata
 >>> unicodedata.name(b"\xb6".decode("ISO-8859-7"))
'GREEK CAPITAL LETTER ALPHA WITH TONOS'

You'll need to find out where that bytestring is coming from.

0
python3897 (1417)
7/4/2013 11:06:40 AM
Στις 4/7/2013 1:54 μμ, ο/η Chris Angelico έγραψε:
> On Thu, Jul 4, 2013 at 8:38 PM, ����� <nikos@superhost.gr> wrote:
>> So you are also suggesting that what gesthostbyaddr() returns is not utf-8
>> encoded too?
>>
>> What character is 0xb6 anyways?
>
> It isn't. It's a byte. Bytes are not characters.
>
> http://www.joelonsoftware.com/articles/Unicode.html

Well in case of utf-8 encoding for the first 127 codepoing we can safely 
say that a character equals a byte :)

-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/4/2013 11:29:22 AM
Στις 4/7/2013 2:06 μμ, ο/η MRAB έγραψε:
> On 04/07/2013 11:38, Νίκος wrote:
>> Στις 4/7/2013 12:50 μμ, ο/η Ulrich Eckhardt έγραψε:
>>> Am 04.07.2013 10:37, schrieb Νίκος:
>>>> I just started to have this error without changing nothing
>>>
>>> Well, undo the nothing that you didn't change. ;)
>>>
>>>> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0:
>>>> invalid start byte
>>>> [Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97] Premature
>>>> end
>>>> of script headers: metrites.py
>>>>
>>>> Why cant it decode the starting byte? what starting byte is that?
>>>
>>> It's the 0xb6 but it's expecting the starting byte of a UTF-8 sequence.
>>> Please do some research on UTF-8, that should clear it up. You could
>>> also search for common causes of that error.
>>
>> So you are also suggesting that what gesthostbyaddr() returns is not
>> utf-8 encoded too?
>>
>> What character is 0xb6 anyways?
>>
> Well, it's from a bytestring, so you'll have to specify what encoding
> you're using! (It clearly isn't UTF-8.)
>
> If it's ISO-8859-7 (what you've previously referred to as "greek-iso"),
> then:
>
>  >>> import unicodedata
>  >>> unicodedata.name(b"\xb6".decode("ISO-8859-7"))
> 'GREEK CAPITAL LETTER ALPHA WITH TONOS'
>
> You'll need to find out where that bytestring is coming from.

Right.
But nowhere in my script(metrites.py) i use an 'Ά' so i really have no 
clue where this is coming from.

And you are right if it was a byte came from an utf-8 encoding scheme 
then it would be automatically decoded.

The only thing i can say for use is that this problem a[[ear only when i 
cloudflare my domain "superhost.gr"

If i un-cloudlflare it it cease to display errors.

Can you tell me hpw to write the following properly:

host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or 'UnResolved'

so even if the function fails "unresolved" to be returned back?
Somehow i need to capture the error.

Or it dosnt have to do it the or operand will be returned?



-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/4/2013 11:36:25 AM
On 04/07/2013 12:29, Νίκος wrote:
> Στις 4/7/2013 1:54 μμ, ο/η Chris Angelico έγραψε:
>> On Thu, Jul 4, 2013 at 8:38 PM, ����� <nikos@superhost.gr> wrote:
>>> So you are also suggesting that what gesthostbyaddr() returns is not utf-8
>>> encoded too?
>>>
>>> What character is 0xb6 anyways?
>>
>> It isn't. It's a byte. Bytes are not characters.
>>
>> http://www.joelonsoftware.com/articles/Unicode.html
>
> Well in case of utf-8 encoding for the first 127 codepoing we can safely
> say that a character equals a byte :)
>
Equals? No. Bytes are not characters. (Strictly speaking, they're
codepoints, not characters.)

And anyway, it's the first _128_ codepoints.
0
python3897 (1417)
7/4/2013 11:52:29 AM
Στις 4/7/2013 2:52 μμ, ο/η MRAB έγραψε:
> On 04/07/2013 12:29, Νίκος wrote:
>> Στις 4/7/2013 1:54 μμ, ο/η Chris Angelico έγραψε:
>>> On Thu, Jul 4, 2013 at 8:38 PM, ����� <nikos@superhost.gr> wrote:
>>>> So you are also suggesting that what gesthostbyaddr() returns is not
>>>> utf-8
>>>> encoded too?
>>>>
>>>> What character is 0xb6 anyways?
>>>
>>> It isn't. It's a byte. Bytes are not characters.
>>>
>>> http://www.joelonsoftware.com/articles/Unicode.html
>>
>> Well in case of utf-8 encoding for the first 127 codepoing we can safely
>> say that a character equals a byte :)
>>
> Equals? No. Bytes are not characters. (Strictly speaking, they're
> codepoints, not characters.)
>
> And anyway, it's the first _128_ codepoints.

Yes 0-127 = 128, i knew that!

Well the relationship between characters and bytes is that:

A [0-127] Unicode codepoints(characters) need 1-byte to be stored in 
utf-8 encoding.

I think its also correct to say that the byte in the above situation is 
the representation of our character.

-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/4/2013 12:06:47 PM
On 04/07/2013 12:36, Νίκος wrote:
> Στις 4/7/2013 2:06 μμ, ο/η MRAB έγραψε:
>> On 04/07/2013 11:38, Νίκος wrote:
>>> Στις 4/7/2013 12:50 μμ, ο/η Ulrich Eckhardt έγραψε:
>>>> Am 04.07.2013 10:37, schrieb Νίκος:
>>>>> I just started to have this error without changing nothing
>>>>
>>>> Well, undo the nothing that you didn't change. ;)
>>>>
>>>>> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0:
>>>>> invalid start byte
>>>>> [Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97] Premature
>>>>> end
>>>>> of script headers: metrites.py
>>>>>
>>>>> Why cant it decode the starting byte? what starting byte is that?
>>>>
>>>> It's the 0xb6 but it's expecting the starting byte of a UTF-8 sequence.
>>>> Please do some research on UTF-8, that should clear it up. You could
>>>> also search for common causes of that error.
>>>
>>> So you are also suggesting that what gesthostbyaddr() returns is not
>>> utf-8 encoded too?
>>>
>>> What character is 0xb6 anyways?
>>>
>> Well, it's from a bytestring, so you'll have to specify what encoding
>> you're using! (It clearly isn't UTF-8.)
>>
>> If it's ISO-8859-7 (what you've previously referred to as "greek-iso"),
>> then:
>>
>>  >>> import unicodedata
>>  >>> unicodedata.name(b"\xb6".decode("ISO-8859-7"))
>> 'GREEK CAPITAL LETTER ALPHA WITH TONOS'
>>
>> You'll need to find out where that bytestring is coming from.
>
> Right.
> But nowhere in my script(metrites.py) i use an 'Ά' so i really have no
> clue where this is coming from.
>
> And you are right if it was a byte came from an utf-8 encoding scheme
> then it would be automatically decoded.
>
> The only thing i can say for use is that this problem a[[ear only when i
> cloudflare my domain "superhost.gr"
>
> If i un-cloudlflare it it cease to display errors.
>
> Can you tell me hpw to write the following properly:
>
> host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or 'UnResolved'
>
> so even if the function fails "unresolved" to be returned back?
> Somehow i need to capture the error.
>
> Or it dosnt have to do it the or operand will be returned?
>
If gethostbyaddr fails, it raises socket.gaierror, (which, from Python
3.3 onwards, is a subclass of OSError), so try catching that, setting
'host' to 'UnResolved' if it's raised.

Also, try printing out ascii(os.environ['REMOTE_ADDR']).

0
python3897 (1417)
7/4/2013 12:07:41 PM
On Thu, Jul 4, 2013 at 9:52 PM, MRAB <python@mrabarnett.plus.com> wrote:
> On 04/07/2013 12:29, =CE=9D=CE=AF=CE=BA=CE=BF=CF=82 wrote:
>>
>> =CE=A3=CF=84=CE=B9=CF=82 4/7/2013 1:54 =CE=BC=CE=BC, =CE=BF/=CE=B7 Chris=
 Angelico =CE=AD=CE=B3=CF=81=CE=B1=CF=88=CE=B5:
>>>
>>> On Thu, Jul 4, 2013 at 8:38 PM, =EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=BF=BD=EF=
=BF=BD <nikos@superhost.gr> wrote:
>>>>
>>>> So you are also suggesting that what gesthostbyaddr() returns is not
>>>> utf-8
>>>> encoded too?
>>>>
>>>> What character is 0xb6 anyways?
>>>
>>>
>>> It isn't. It's a byte. Bytes are not characters.
>>>
>>> http://www.joelonsoftware.com/articles/Unicode.html
>>
>>
>> Well in case of utf-8 encoding for the first 127 codepoing we can safely
>> say that a character equals a byte :)
>>
> Equals? No. Bytes are not characters. (Strictly speaking, they're
> codepoints, not characters.)
>
> And anyway, it's the first _128_ codepoints.

As MRAB says, even if there's a 1:1 correspondence between bytes,
codepoints, and characters, they're still not the same thing. Plus,
0xb6 is not in the first 128, so your statement is false and your
question has no answer. Do you understand why I gave you that link? If
not, go read the page linked to.

ChrisA
0
rosuav (4324)
7/4/2013 12:37:40 PM
Στις 4/7/2013 3:07 μμ, ο/η MRAB έγραψε:
> On 04/07/2013 12:36, Νίκος wrote:
>> Στις 4/7/2013 2:06 μμ, ο/η MRAB έγραψε:
>>> On 04/07/2013 11:38, Νίκος wrote:
>>>> Στις 4/7/2013 12:50 μμ, ο/η Ulrich Eckhardt έγραψε:
>>>>> Am 04.07.2013 10:37, schrieb Νίκος:
>>>>>> I just started to have this error without changing nothing
>>>>>
>>>>> Well, undo the nothing that you didn't change. ;)
>>>>>
>>>>>> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in
>>>>>> position 0:
>>>>>> invalid start byte
>>>>>> [Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97] Premature
>>>>>> end
>>>>>> of script headers: metrites.py
>>>>>>
>>>>>> Why cant it decode the starting byte? what starting byte is that?
>>>>>
>>>>> It's the 0xb6 but it's expecting the starting byte of a UTF-8
>>>>> sequence.
>>>>> Please do some research on UTF-8, that should clear it up. You could
>>>>> also search for common causes of that error.
>>>>
>>>> So you are also suggesting that what gesthostbyaddr() returns is not
>>>> utf-8 encoded too?
>>>>
>>>> What character is 0xb6 anyways?
>>>>
>>> Well, it's from a bytestring, so you'll have to specify what encoding
>>> you're using! (It clearly isn't UTF-8.)
>>>
>>> If it's ISO-8859-7 (what you've previously referred to as "greek-iso"),
>>> then:
>>>
>>>  >>> import unicodedata
>>>  >>> unicodedata.name(b"\xb6".decode("ISO-8859-7"))
>>> 'GREEK CAPITAL LETTER ALPHA WITH TONOS'
>>>
>>> You'll need to find out where that bytestring is coming from.
>>
>> Right.
>> But nowhere in my script(metrites.py) i use an 'Ά' so i really have no
>> clue where this is coming from.
>>
>> And you are right if it was a byte came from an utf-8 encoding scheme
>> then it would be automatically decoded.
>>
>> The only thing i can say for use is that this problem a[[ear only when i
>> cloudflare my domain "superhost.gr"
>>
>> If i un-cloudlflare it it cease to display errors.
>>
>> Can you tell me hpw to write the following properly:
>>
>> host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or
>> 'UnResolved'
>>
>> so even if the function fails "unresolved" to be returned back?
>> Somehow i need to capture the error.
>>
>> Or it dosnt have to do it the or operand will be returned?
>>
> If gethostbyaddr fails, it raises socket.gaierror, (which, from Python
> 3.3 onwards, is a subclass of OSError), so try catching that, setting
> 'host' to 'UnResolved' if it's raised.
>
> Also, try printing out ascii(os.environ['REMOTE_ADDR']).
>

I have followed your suggestion by trying this:

try:
	host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
except socket.gaierror:
	host = "UnResolved"

and then re-cloudlflared "superhost.gr" domain

http://superhost.gr/ gives internal server error.

-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/4/2013 12:47:23 PM
Στις 4/7/2013 3:07 μμ, ο/η MRAB έγραψε:
> Also, try printing out ascii(os.environ['REMOTE_ADDR']).

'108.162.229.97' is the result of:

print( ascii(os.environ['REMOTE_ADDR']) )

Seems perfectly valid. and also have a PTR record, so that leaved us 
clueless about the internal server error.
-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/4/2013 12:52:59 PM
Am 04.07.2013 12:38, schrieb Νίκος:
> Στις 4/7/2013 12:50 μμ, ο/η Ulrich Eckhardt έγραψε:
>> Am 04.07.2013 10:37, schrieb Νίκος:
>>> Why cant it decode the starting byte? what starting byte is that?
>>
>> It's the 0xb6 but it's expecting the starting byte of a UTF-8 sequence.
>> Please do some research on UTF-8, that should clear it up. You could
>> also search for common causes of that error.
>
> So you are also suggesting that what gesthostbyaddr() returns is not
> utf-8 encoded too?

I never said that. And do some research yourself, you were given plenty 
of hints.

Uli

0
7/4/2013 1:07:11 PM
Στις 4/7/2013 4:07 μμ, ο/η Ulrich Eckhardt έγραψε:
> Am 04.07.2013 12:38, schrieb Νίκος:
>> Στις 4/7/2013 12:50 μμ, ο/η Ulrich Eckhardt έγραψε:
>>> Am 04.07.2013 10:37, schrieb Νίκος:
>>>> Why cant it decode the starting byte? what starting byte is that?
>>>
>>> It's the 0xb6 but it's expecting the starting byte of a UTF-8 sequence.
>>> Please do some research on UTF-8, that should clear it up. You could
>>> also search for common causes of that error.
>>
>> So you are also suggesting that what gesthostbyaddr() returns is not
>> utf-8 encoded too?
>
> I never said that. And do some research yourself, you were given plenty
> of hints.
>
> Uli
>
Yes and as you can see form my responses i have tried any suggestion so 
far and the problem still remains unresolved.

What you said implied that the string returned by the function cannot be 
decoded as utf-8.

-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/4/2013 1:26:30 PM
On 04/07/2013 13:47, Νίκος wrote:
> Στις 4/7/2013 3:07 μμ, ο/η MRAB έγραψε:
>> On 04/07/2013 12:36, Νίκος wrote:
>>> Στις 4/7/2013 2:06 μμ, ο/η MRAB έγραψε:
>>>> On 04/07/2013 11:38, Νίκος wrote:
>>>>> Στις 4/7/2013 12:50 μμ, ο/η Ulrich Eckhardt έγραψε:
>>>>>> Am 04.07.2013 10:37, schrieb Νίκος:
>>>>>>> I just started to have this error without changing nothing
>>>>>>
>>>>>> Well, undo the nothing that you didn't change. ;)
>>>>>>
>>>>>>> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in
>>>>>>> position 0:
>>>>>>> invalid start byte
>>>>>>> [Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97] Premature
>>>>>>> end
>>>>>>> of script headers: metrites.py
>>>>>>>
>>>>>>> Why cant it decode the starting byte? what starting byte is that?
>>>>>>
>>>>>> It's the 0xb6 but it's expecting the starting byte of a UTF-8
>>>>>> sequence.
>>>>>> Please do some research on UTF-8, that should clear it up. You could
>>>>>> also search for common causes of that error.
>>>>>
>>>>> So you are also suggesting that what gesthostbyaddr() returns is not
>>>>> utf-8 encoded too?
>>>>>
>>>>> What character is 0xb6 anyways?
>>>>>
>>>> Well, it's from a bytestring, so you'll have to specify what encoding
>>>> you're using! (It clearly isn't UTF-8.)
>>>>
>>>> If it's ISO-8859-7 (what you've previously referred to as "greek-iso"),
>>>> then:
>>>>
>>>>  >>> import unicodedata
>>>>  >>> unicodedata.name(b"\xb6".decode("ISO-8859-7"))
>>>> 'GREEK CAPITAL LETTER ALPHA WITH TONOS'
>>>>
>>>> You'll need to find out where that bytestring is coming from.
>>>
>>> Right.
>>> But nowhere in my script(metrites.py) i use an 'Ά' so i really have no
>>> clue where this is coming from.
>>>
>>> And you are right if it was a byte came from an utf-8 encoding scheme
>>> then it would be automatically decoded.
>>>
>>> The only thing i can say for use is that this problem a[[ear only when i
>>> cloudflare my domain "superhost.gr"
>>>
>>> If i un-cloudlflare it it cease to display errors.
>>>
>>> Can you tell me hpw to write the following properly:
>>>
>>> host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or
>>> 'UnResolved'
>>>
>>> so even if the function fails "unresolved" to be returned back?
>>> Somehow i need to capture the error.
>>>
>>> Or it dosnt have to do it the or operand will be returned?
>>>
>> If gethostbyaddr fails, it raises socket.gaierror, (which, from Python
>> 3.3 onwards, is a subclass of OSError), so try catching that, setting
>> 'host' to 'UnResolved' if it's raised.
>>
>> Also, try printing out ascii(os.environ['REMOTE_ADDR']).
>>
>
> I have followed your suggestion by trying this:
>
> try:
> 	host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
> except socket.gaierror:
> 	host = "UnResolved"
>
> and then re-cloudlflared "superhost.gr" domain
>
> http://superhost.gr/ gives internal server error.
>
Try catching OSError instead. (As I said, from Python 3.3,
socket.gaierror is a subclass of it.)

0
python3897 (1417)
7/4/2013 1:34:39 PM
On 04/07/2013 13:52, Νίκος wrote:
> Στις 4/7/2013 3:07 μμ, ο/η MRAB έγραψε:
>> Also, try printing out ascii(os.environ['REMOTE_ADDR']).
>
> '108.162.229.97' is the result of:
>
> print( ascii(os.environ['REMOTE_ADDR']) )
>
> Seems perfectly valid. and also have a PTR record, so that leaved us
> clueless about the internal server error.
>
For me, socket.gethostbyaddr('108.162.229.97') raises socket.herror,
which is also a subclass of OSError from Python 3.3 onwards.

0
python3897 (1417)
7/4/2013 1:34:42 PM
Στις 4/7/2013 4:34 μμ, ο/η MRAB έγραψε:
> On 04/07/2013 13:47, Νίκος wrote:
>> Στις 4/7/2013 3:07 μμ, ο/η MRAB έγραψε:
>>> On 04/07/2013 12:36, Νίκος wrote:
>>>> Στις 4/7/2013 2:06 μμ, ο/η MRAB έγραψε:
>>>>> On 04/07/2013 11:38, Νίκος wrote:
>>>>>> Στις 4/7/2013 12:50 μμ, ο/η Ulrich Eckhardt έγραψε:
>>>>>>> Am 04.07.2013 10:37, schrieb Νίκος:
>>>>>>>> I just started to have this error without changing nothing
>>>>>>>
>>>>>>> Well, undo the nothing that you didn't change. ;)
>>>>>>>
>>>>>>>> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in
>>>>>>>> position 0:
>>>>>>>> invalid start byte
>>>>>>>> [Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97]
>>>>>>>> Premature
>>>>>>>> end
>>>>>>>> of script headers: metrites.py
>>>>>>>>
>>>>>>>> Why cant it decode the starting byte? what starting byte is that?
>>>>>>>
>>>>>>> It's the 0xb6 but it's expecting the starting byte of a UTF-8
>>>>>>> sequence.
>>>>>>> Please do some research on UTF-8, that should clear it up. You could
>>>>>>> also search for common causes of that error.
>>>>>>
>>>>>> So you are also suggesting that what gesthostbyaddr() returns is not
>>>>>> utf-8 encoded too?
>>>>>>
>>>>>> What character is 0xb6 anyways?
>>>>>>
>>>>> Well, it's from a bytestring, so you'll have to specify what encoding
>>>>> you're using! (It clearly isn't UTF-8.)
>>>>>
>>>>> If it's ISO-8859-7 (what you've previously referred to as
>>>>> "greek-iso"),
>>>>> then:
>>>>>
>>>>>  >>> import unicodedata
>>>>>  >>> unicodedata.name(b"\xb6".decode("ISO-8859-7"))
>>>>> 'GREEK CAPITAL LETTER ALPHA WITH TONOS'
>>>>>
>>>>> You'll need to find out where that bytestring is coming from.
>>>>
>>>> Right.
>>>> But nowhere in my script(metrites.py) i use an 'Ά' so i really have no
>>>> clue where this is coming from.
>>>>
>>>> And you are right if it was a byte came from an utf-8 encoding scheme
>>>> then it would be automatically decoded.
>>>>
>>>> The only thing i can say for use is that this problem a[[ear only
>>>> when i
>>>> cloudflare my domain "superhost.gr"
>>>>
>>>> If i un-cloudlflare it it cease to display errors.
>>>>
>>>> Can you tell me hpw to write the following properly:
>>>>
>>>> host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or
>>>> 'UnResolved'
>>>>
>>>> so even if the function fails "unresolved" to be returned back?
>>>> Somehow i need to capture the error.
>>>>
>>>> Or it dosnt have to do it the or operand will be returned?
>>>>
>>> If gethostbyaddr fails, it raises socket.gaierror, (which, from Python
>>> 3.3 onwards, is a subclass of OSError), so try catching that, setting
>>> 'host' to 'UnResolved' if it's raised.
>>>
>>> Also, try printing out ascii(os.environ['REMOTE_ADDR']).
>>>
>>
>> I have followed your suggestion by trying this:
>>
>> try:
>>     host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
>> except socket.gaierror:
>>     host = "UnResolved"
>>
>> and then re-cloudlflared "superhost.gr" domain
>>
>> http://superhost.gr/ gives internal server error.
>>
> Try catching OSError instead. (As I said, from Python 3.3,
> socket.gaierror is a subclass of it.)
>

At least CloudFlare doesn't give me issues:

if i try this:

try:
	host = os.environ['REMOTE_ADDR'][0]
except socket.gaierror:
	host = "UnResolved"

then i get no errors and a valid ip back

but the above fails.

I don't know how to catch the exception with OSError.

i know only this two:

except socket.gaierror:
except socket.herror

both fail.

-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/4/2013 1:38:35 PM
On 04/07/2013 14:38, Νίκος Γκρ33κ wrote:
> Στις 4/7/2013 4:34 μμ, ο/η MRAB έγραψε:
>> On 04/07/2013 13:47, Νίκος wrote:
>>> Στις 4/7/2013 3:07 μμ, ο/η MRAB έγραψε:
>>>> On 04/07/2013 12:36, Νίκος wrote:
>>>>> Στις 4/7/2013 2:06 μμ, ο/η MRAB έγραψε:
>>>>>> On 04/07/2013 11:38, Νίκος wrote:
>>>>>>> Στις 4/7/2013 12:50 μμ, ο/η Ulrich Eckhardt έγραψε:
>>>>>>>> Am 04.07.2013 10:37, schrieb Νίκος:
>>>>>>>>> I just started to have this error without changing nothing
>>>>>>>>
>>>>>>>> Well, undo the nothing that you didn't change. ;)
>>>>>>>>
>>>>>>>>> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in
>>>>>>>>> position 0:
>>>>>>>>> invalid start byte
>>>>>>>>> [Thu Jul 04 11:35:14 2013] [error] [client 108.162.229.97]
>>>>>>>>> Premature
>>>>>>>>> end
>>>>>>>>> of script headers: metrites.py
>>>>>>>>>
>>>>>>>>> Why cant it decode the starting byte? what starting byte is that?
>>>>>>>>
>>>>>>>> It's the 0xb6 but it's expecting the starting byte of a UTF-8
>>>>>>>> sequence.
>>>>>>>> Please do some research on UTF-8, that should clear it up. You could
>>>>>>>> also search for common causes of that error.
>>>>>>>
>>>>>>> So you are also suggesting that what gesthostbyaddr() returns is not
>>>>>>> utf-8 encoded too?
>>>>>>>
>>>>>>> What character is 0xb6 anyways?
>>>>>>>
>>>>>> Well, it's from a bytestring, so you'll have to specify what encoding
>>>>>> you're using! (It clearly isn't UTF-8.)
>>>>>>
>>>>>> If it's ISO-8859-7 (what you've previously referred to as
>>>>>> "greek-iso"),
>>>>>> then:
>>>>>>
>>>>>>  >>> import unicodedata
>>>>>>  >>> unicodedata.name(b"\xb6".decode("ISO-8859-7"))
>>>>>> 'GREEK CAPITAL LETTER ALPHA WITH TONOS'
>>>>>>
>>>>>> You'll need to find out where that bytestring is coming from.
>>>>>
>>>>> Right.
>>>>> But nowhere in my script(metrites.py) i use an 'Ά' so i really have no
>>>>> clue where this is coming from.
>>>>>
>>>>> And you are right if it was a byte came from an utf-8 encoding scheme
>>>>> then it would be automatically decoded.
>>>>>
>>>>> The only thing i can say for use is that this problem a[[ear only
>>>>> when i
>>>>> cloudflare my domain "superhost.gr"
>>>>>
>>>>> If i un-cloudlflare it it cease to display errors.
>>>>>
>>>>> Can you tell me hpw to write the following properly:
>>>>>
>>>>> host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or
>>>>> 'UnResolved'
>>>>>
>>>>> so even if the function fails "unresolved" to be returned back?
>>>>> Somehow i need to capture the error.
>>>>>
>>>>> Or it dosnt have to do it the or operand will be returned?
>>>>>
>>>> If gethostbyaddr fails, it raises socket.gaierror, (which, from Python
>>>> 3.3 onwards, is a subclass of OSError), so try catching that, setting
>>>> 'host' to 'UnResolved' if it's raised.
>>>>
>>>> Also, try printing out ascii(os.environ['REMOTE_ADDR']).
>>>>
>>>
>>> I have followed your suggestion by trying this:
>>>
>>> try:
>>>     host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
>>> except socket.gaierror:
>>>     host = "UnResolved"
>>>
>>> and then re-cloudlflared "superhost.gr" domain
>>>
>>> http://superhost.gr/ gives internal server error.
>>>
>> Try catching OSError instead. (As I said, from Python 3.3,
>> socket.gaierror is a subclass of it.)
>>
>
> At least CloudFlare doesn't give me issues:
>
> if i try this:
>
> try:
> 	host = os.environ['REMOTE_ADDR'][0]
> except socket.gaierror:
> 	host = "UnResolved"
>
It's pointless trying to catch a socket exception here because you're
not using a socket, you're just getting a string from an environment
variable.

> then i get no errors and a valid ip back
>
> but the above fails.
>
> I don't know how to catch the exception with OSError.
>
> i know only this two:
>
> except socket.gaierror:
> except socket.herror
>
> both fail.
>
What do you mean "I don't know how to catch the exception with
OSError"? You've tried "except socket.gaierror" and "except
socket.herror", well just write "except OSError" instead!

0
python3897 (1417)
7/4/2013 3:10:56 PM
Στις 4/7/2013 6:10 μμ, ο/η MRAB έγραψε:
> What do you mean "I don't know how to catch the exception with
> OSError"? You've tried "except socket.gaierror" and "except
> socket.herror", well just write "except OSError" instead!


try:
	host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
except OSError:
	host = "UnResolved"

produces also an internal server error.

Are you sure is just except OSError ?

seems very general...

-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/4/2013 3:56:14 PM
On Thu, 04 Jul 2013 13:38:09 +0300, ????? <nikos@superhost.gr> declaimed
the following:

>What character is 0xb6 anyways?
>
	It depends on the encoding... In EBCDIC it's unassigned. It's a
paragraph mark in ISO-Latin-1 (ISO-8859-1). Apparently also a paragraph
mark in ISO-Latin-9 (ISO-8859-15).

	If it is valid in UTF-8, I can't find a reference. It's not a prefix
for a multi-byte character, which implies that the previous byte should
have been something in prefix or another extended byte entry...



-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

0
wlfraed (4596)
7/4/2013 8:08:55 PM
Στις 4/7/2013 11:08 μμ, ο/η Dennis Lee Bieber έγραψε:
> On Thu, 04 Jul 2013 13:38:09 +0300, ????? <nikos@superhost.gr> declaimed
> the following:
>
>> What character is 0xb6 anyways?
>>
> 	It depends on the encoding... In EBCDIC it's unassigned. It's a
> paragraph mark in ISO-Latin-1 (ISO-8859-1). Apparently also a paragraph
> mark in ISO-Latin-9 (ISO-8859-15).
>
> 	If it is valid in UTF-8, I can't find a reference. It's not a prefix
> for a multi-byte character, which implies that the previous byte should
> have been something in prefix or another extended byte entry...

try:
	host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
except:
	host = "Reverse DNS Failed"

Is there a way to write the above so i cna print the error return when 
it fails?

-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/4/2013 8:25:15 PM
Ferrous Cranus <nikos@superhost.gr> writes:

> try:
> 	host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
> except:
> 	host = "Reverse DNS Failed"
>
> Is there a way to write the above so i cna print the error return when
> it fails?

Try something like

try:
	host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
except Exception as e:
	host = "Reverse DNS Failed"
        print(e)

?

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
lele@metapensiero.it  |                 -- Fortunato Depero, 1929.

0
lele4264 (70)
7/4/2013 8:44:10 PM
On 07/04/2013 02:25 PM, Ferrous Cranus wrote:
> try:
> 	host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
> except:
> 	host = "Reverse DNS Failed"
> 
> Is there a way to write the above so i cna print the error return when 
> it fails?
> 

Do you know what IP address causes the failure?  If so write a little
python program that does the socket.gethostbyaddr and run it on the
command line!  Debugging through the CGI interface sucks.

Have you been writing python tests that you can run on the command line
during your development these last weeks?
0
torriem1 (469)
7/4/2013 8:53:34 PM
On Thu, 04 Jul 2013 13:38:09 +0300, Νίκος wrote:

> So you are also suggesting that what gesthostbyaddr() returns is not 
> utf-8 encoded too?

The gethostbyaddr() OS function returns a byte string with no specified
encoding. Python 3 will doubtless try to decode that to a character string
using some (probably unspecified) encoding.

Names obtained from DNS should consist entirely of ASCII characters
(gethostbyname shouldn't attempt to decode internationalised names
which use IDN, it should return the raw data).

Names obtained by other means (e.g. /etc/hosts or Active Directory) could
contain anything, but if you use non-ASCII hostnames you're asking for
trouble.

0
nobody (5159)
7/5/2013 12:06:42 AM
Στις 5/7/2013 3:06 πμ, ο/η Nobody έγραψε:
> On Thu, 04 Jul 2013 13:38:09 +0300, Νίκος wrote:
>
>> So you are also suggesting that what gesthostbyaddr() returns is not
>> utf-8 encoded too?
>
> The gethostbyaddr() OS function returns a byte string with no specified
> encoding. Python 3 will doubtless try to decode that to a character string
> using some (probably unspecified) encoding.

I see, but if the function returns a byte string not inutf-8 format then 
how my script is uspposes to decode this byte string?

And why only this error happens when i cloudflare my domain, while when 
i un-cloudflare it are reverse DNS are being resolves without problem.
So the queston is: How come it only fails when i cloidflare the domain?

Also please comment on that:

host =  gethostbyaddr(....) or "UnResolved"

This will return the first argument that define the evaluation as being 
true or untrue.

if function returns false the the 2nd argument.
Nut if the function gives an exception will the condition return the 2nd 
argument or will the program fail?

I was udner the impression that i could avoid error handling  inside 
try/excepts by utilizing "or".
-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/5/2013 6:07:19 AM
Στις 5/7/2013 3:06 πμ, ο/η Nobody έγραψε:
> On Thu, 04 Jul 2013 13:38:09 +0300, Νίκος wrote:
>
>> So you are also suggesting that what gesthostbyaddr() returns is not
>> utf-8 encoded too?
>
> The gethostbyaddr() OS function returns a byte string with no specified
> encoding. Python 3 will doubtless try to decode that to a character string
> using some (probably unspecified) encoding.
>
> Names obtained from DNS should consist entirely of ASCII characters
> (gethostbyname shouldn't attempt to decode internationalised names
> which use IDN, it should return the raw data).
>
> Names obtained by other means (e.g. /etc/hosts or Active Directory) could
> contain anything, but if you use non-ASCII hostnames you're asking for
> trouble.
>

Please help because i just happened to noticed that after having this code:

try:
	host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
except Exception as e:
	host = "Reverse DNS Failed"


all requests are being resolves, result to:


Reverse DNS Failed as you can see here:
http://superhost.gr/?show=log&page=index.html

How can the above code not be able to reeverse dns any more and it falls 
back to  the failed string?

-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/5/2013 6:51:27 AM
Ferrous Cranus <nikos@superhost.gr> writes:

> host =  gethostbyaddr(....) or "UnResolved"
>
> This will return the first argument that define the evaluation as
> being true or untrue.
>
> if function returns false the the 2nd argument.
> Nut if the function gives an exception will the condition return the
> 2nd argument or will the program fail?
>
> I was udner the impression that i could avoid error handling  inside
> try/excepts by utilizing "or".

No, you had the wrong impression. Why don't you simply invoke the Python
interpreter and try things out with that??

    >>> a = 1/0 or 100
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ZeroDivisionError: division by zero
    >>> a
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    NameError: name 'a' is not defined
    >>> a = 0/1 or 100
    >>> a
    100
    >>>

ciao, lele.
--
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
lele@metapensiero.it  |                 -- Fortunato Depero, 1929.

0
lele4264 (70)
7/5/2013 6:55:20 AM
Νίκος Gr33k <nikos@superhost.gr> writes:

> try:
> 	host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
> except Exception as e:
> 	host = "Reverse DNS Failed"
>
> How can the above code not be able to reeverse dns any more and it
> falls back to  the failed string?

The only way to know is actually printing out the exception, either to
stderr, or better using the logging facility, as I suggested.

FYI, your code above is (almost) exactly equivalent to the simpler

    try:
    	host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
    except:
    	host = "Reverse DNS Failed"

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
lele@metapensiero.it  |                 -- Fortunato Depero, 1929.

0
lele4264 (70)
7/5/2013 7:06:22 AM
Στις 5/7/2013 9:55 πμ, ο/η Lele Gaifax έγραψε:
> Ferrous Cranus <nikos@superhost.gr> writes:
>
>> host =  gethostbyaddr(....) or "UnResolved"
>>
>> This will return the first argument that define the evaluation as
>> being true or untrue.
>>
>> if function returns false the the 2nd argument.
>> Nut if the function gives an exception will the condition return the
>> 2nd argument or will the program fail?
>>
>> I was udner the impression that i could avoid error handling  inside
>> try/excepts by utilizing "or".
>
> No, you had the wrong impression. Why don't you simply invoke the Python
> interpreter and try things out with that??
>
>      >>> a = 1/0 or 100
>      Traceback (most recent call last):
>        File "<stdin>", line 1, in <module>
>      ZeroDivisionError: division by zero
>      >>> a
>      Traceback (most recent call last):
>        File "<stdin>", line 1, in <module>
>      NameError: name 'a' is not defined
>      >>> a = 0/1 or 100
>      >>> a
>      100
>      >>>

Thank you Lele, i wanted to but i had no idea how to test it.
Your devision by zero is very smart thing to test!

So it proves that a condition cannot be evaluation as truthy or falsey 
if one of the operators is giving out an exception.
Thank you.


-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/5/2013 7:10:12 AM
Στις 5/7/2013 10:06 πμ, ο/η Lele Gaifax έγραψε:
>      try:
>      	host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
>      except:
>      	host = "Reverse DNS Failed"


Yes i uses to had it like that, until i was looking for ways to make it 
hold the error

except Exception as e:
	print( e )
	host = e

but print( e ) in the way i used to had it doesn't print out the error, 
it instead gives an internal server error on browser.

I must somehow take a look at the error to understand why every visitor 
i have gets UnResolved, but how since prints fails?

-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/5/2013 7:13:59 AM
--089e0118409ec835f704e0be89b7
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Jul 5, 2013 12:12 AM, "Lele Gaifax" <lele@metapensiero.it> wrote:
>
> =CE=9D=CE=AF=CE=BA=CE=BF=CF=82 Gr33k <nikos@superhost.gr> writes:
>
> > try:
> >       host =3D socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
> > except Exception as e:
> >       host =3D "Reverse DNS Failed"
> >
> > How can the above code not be able to reeverse dns any more and it
> > falls back to  the failed string?
>
> The only way to know is actually printing out the exception, either to
> stderr, or better using the logging facility, as I suggested.
>
> FYI, your code above is (almost) exactly equivalent to the simpler
>
>     try:
>         host =3D socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
>     except:
>         host =3D "Reverse DNS Failed"
>
> ciao, lele.
>

They aren't equivalent. "except Exception" won't catch KeyboardInterrupt or
SystemExit or a few others that you really don't want to catch in a generic
error handler. You should almost never have a bare except.

--089e0118409ec835f704e0be89b7
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<p dir=3D"ltr"><br>
On Jul 5, 2013 12:12 AM, &quot;Lele Gaifax&quot; &lt;<a href=3D"mailto:lele=
@metapensiero.it">lele@metapensiero.it</a>&gt; wrote:<br>
&gt;<br>
&gt; =CE=9D=CE=AF=CE=BA=CE=BF=CF=82 Gr33k &lt;<a href=3D"mailto:nikos@super=
host.gr">nikos@superhost.gr</a>&gt; writes:<br>
&gt;<br>
&gt; &gt; try:<br>
&gt; &gt; =C2=A0 =C2=A0 =C2=A0 host =3D socket.gethostbyaddr( os.environ[&#=
39;REMOTE_ADDR&#39;] )[0]<br>
&gt; &gt; except Exception as e:<br>
&gt; &gt; =C2=A0 =C2=A0 =C2=A0 host =3D &quot;Reverse DNS Failed&quot;<br>
&gt; &gt;<br>
&gt; &gt; How can the above code not be able to reeverse dns any more and i=
t<br>
&gt; &gt; falls back to =C2=A0the failed string?<br>
&gt;<br>
&gt; The only way to know is actually printing out the exception, either to=
<br>
&gt; stderr, or better using the logging facility, as I suggested.<br>
&gt;<br>
&gt; FYI, your code above is (almost) exactly equivalent to the simpler<br>
&gt;<br>
&gt; =C2=A0 =C2=A0 try:<br>
&gt; =C2=A0 =C2=A0 =C2=A0 =C2=A0 host =3D socket.gethostbyaddr( os.environ[=
&#39;REMOTE_ADDR&#39;] )[0]<br>
&gt; =C2=A0 =C2=A0 except:<br>
&gt; =C2=A0 =C2=A0 =C2=A0 =C2=A0 host =3D &quot;Reverse DNS Failed&quot;<br=
>
&gt;<br>
&gt; ciao, lele.<br>
&gt; </p>
<p dir=3D"ltr">They aren&#39;t equivalent. &quot;except Exception&quot; won=
&#39;t catch KeyboardInterrupt or SystemExit or a few others that you reall=
y don&#39;t want to catch in a generic error handler. You should almost nev=
er have a bare except.<br>

</p>

--089e0118409ec835f704e0be89b7--

0
7/5/2013 7:21:06 AM
On 07/05/2013 02:51 AM, Νίκος Gr33k wrote:

     <SNIP>
>
> Please help because i just happened to noticed that after having this code:
>
> try:
>      host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
> except Exception as e:
>      host = "Reverse DNS Failed"

Don't ever catch a bare Exception class.  Make it more specific to the 
particular problem you're trying to suppress.

In particular, your previous problem with the utf-8 decoding will also 
be caught by the Exception class, so it'll get all lumped together as 
"Reverse DNS Failed".

>
>
> all requests are being resolves, result to:
>
>
> Reverse DNS Failed as you can see here:
> http://superhost.gr/?show=log&page=index.html
>
> How can the above code not be able to reeverse dns any more and it falls
> back to  the failed string?
>

Since you've not made any progress with all the other suggestions, how 
about if you decompose this line into several, and see just which one is 
failing?  Maybe that'll tell you what's going on.  In general, 
suppressing an exception without knowing why it's firing is a huge mistake.

The line started as:

 >      host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]

refactor that to:
        remadd = os.environ('REMOVE_ADDR')
        tuple3 = socket.gethostbyaddr(remadd)
        host = tuple3[0]

and see which one throws the exception.  Then once you have that, 
examine the exact parameters that might be triggering the problem.  In 
particular, figure out the exact types and values for remadd and tuple3.

        print(type(remadd) + " : " + repr(remadd))

Of course, print itself won't work too well in a CGI environment.  But 
you must have solved that problem by now, either using log files or 
running the program excerpt in a regular console.


-- 
DaveA

0
davea9629 (629)
7/5/2013 7:50:52 AM
Στις 5/7/2013 10:50 πμ, ο/η Dave Angel έγραψε:
>
> The line started as:
>
>  >      host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
>
> refactor that to:
>         remadd = os.environ('REMOVE_ADDR')
>         tuple3 = socket.gethostbyaddr(remadd)
>         host = tuple3[0]
>
> and see which one throws the exception.  Then once you have that,
> examine the exact parameters that might be triggering the problem.  In
> particular, figure out the exact types and values for remadd and tuple3.
>
>         print(type(remadd) + " : " + repr(remadd))

I'am not sure how iam supposed to write this: i just tried this:


try:
	remadd = os.environ('REMOVE_ADDR')
	tuple3 = socket.gethostbyaddr(remadd)
	host = tuple3[0]
except:
	host = type(remadd) + " : " + repr(remadd)


but iam getting an internal server error.

I didnt print it as you said but its the same thing host var gets 
printed later on.

Now, why would this give an internal server error?

-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/5/2013 8:00:21 AM
On 07/05/2013 03:13 AM, Νίκος Gr33k wrote:
> Στις 5/7/2013 10:06 πμ, ο/η Lele Gaifax έγραψε:
>>      try:
>>          host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
>>      except:
>>          host = "Reverse DNS Failed"
>
>
> Yes i uses to had it like that, until i was looking for ways to make it
> hold the error
>
> except Exception as e:
>      print( e )
>      host = e
>
> but print( e ) in the way i used to had it doesn't print out the error,
> it instead gives an internal server error on browser.
>
> I must somehow take a look at the error to understand why every visitor
> i have gets UnResolved, but how since prints fails?
>


How have you been doing it all along?  Just open a console onto that 
server, start the appropriate version of Python interactively, and try 
the things we've been talking about.  If it fails the same way as within 
the cgi environmnet, you get full visibility.

Or if the problems cannot be recreated outside the cgi environment, use 
the log files you've been logging other problems into.  Or simply open a 
text file for writing, and add a file= keyword parameter to the print 
function call.

      print(repr(e), file=myfile)



-- 
DaveA

0
davea9629 (629)
7/5/2013 8:27:23 AM
On 07/05/2013 04:00 AM, Νίκος Gr33k wrote:
> Στις 5/7/2013 10:50 πμ, ο/η Dave Angel έγραψε:
>>
>> The line started as:
>>
>>  >      host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
>>
>> refactor that to:
>>         remadd = os.environ('REMOVE_ADDR')
>>         tuple3 = socket.gethostbyaddr(remadd)
>>         host = tuple3[0]
>>
>> and see which one throws the exception.  Then once you have that,
>> examine the exact parameters that might be triggering the problem.  In
>> particular, figure out the exact types and values for remadd and tuple3.
>>
>>         print(type(remadd) + " : " + repr(remadd))
>
> I'am not sure how iam supposed to write this: i just tried this:
>
>
> try:
>      remadd = os.environ('REMOVE_ADDR')
>      tuple3 = socket.gethostbyaddr(remadd)
>      host = tuple3[0]
> except:
>      host = type(remadd) + " : " + repr(remadd)
>
>
> but iam getting an internal server error.
>
> I didnt print it as you said but its the same thing host var gets
> printed later on.
>
> Now, why would this give an internal server error?
>

I have no idea what causes an internal server error.  It's up to you to 
get the output of the expression to some location you can examine. 
Easiest way is to run those 3 lines directly on the server, not in the 
cgi environment.

But if you don't have any debugging tools, then STOP right now and build 
some.  Use logging, or redirect print, or do something that the server 
folks provide as debugging aids.  But just blindly guessing is 
ludicrous.  So also is throwing out clues by using a bare except.

Assigning that string to host makes no sense at all.  And neither does 
putting it in the except clause.  You want to get that string to YOUR 
eyes, not to the server who might get an internal server error.


-- 
DaveA

0
davea9629 (629)
7/5/2013 8:35:18 AM
Am Freitag, 5. Juli 2013 10:00:21 UTC+2 schrieb =CE=9D=CE=AF=CE=BA=CE=BF=CF=
=82 Gr33k:
> ...
> I'am not sure how iam supposed to write this: i just tried this:
> try:
> 	remadd =3D os.environ('REMOVE_ADDR')
> 	tuple3 =3D socket.gethostbyaddr(remadd)
> 	host =3D tuple3[0]
> except:
> 	host =3D type(remadd) + " : " + repr(remadd)

Hey, if no one told you before:
You are allowed to read what other people suggest you to do, think about it=
  and so correct obvious typos: 'REMO*V*E_ADDR'

By the way, my i cite:
Am Donnerstag, 4. Juli 2013 14:52:59 UTC+2 schrieb =CE=9D=CE=AF=CE=BA=CE=BF=
=CF=82 Gr33k:
> ...
> '108.162.229.97' is the result of:
> print( ascii(os.environ['REMOTE_ADDR']) )

Am Donnerstag, 4. Juli 2013 16:48 UTC+2 schrieb =CE=9D=CE=AF=CE=BA=CE=BF=CF=
=82 Gr33k:
> For me, socket.gethostbyaddr('108.162.229.97') raises socket.herror,
> which is also a subclass of OSError from Python 3.3 onwards.
0
7/5/2013 8:44:29 AM
Στις 5/7/2013 11:35 πμ, ο/η Dave Angel έγραψε:
> On 07/05/2013 04:00 AM, Νίκος Gr33k wrote:
>> Στις 5/7/2013 10:50 πμ, ο/η Dave Angel έγραψε:
>>>
>>> The line started as:
>>>
>>>  >      host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
>>>
>>> refactor that to:
>>>         remadd = os.environ('REMOVE_ADDR')
>>>         tuple3 = socket.gethostbyaddr(remadd)
>>>         host = tuple3[0]
>>>
>>> and see which one throws the exception.  Then once you have that,
>>> examine the exact parameters that might be triggering the problem.  In
>>> particular, figure out the exact types and values for remadd and tuple3.
>>>
>>>         print(type(remadd) + " : " + repr(remadd))
>>
>> I'am not sure how iam supposed to write this: i just tried this:
>>
>>
>> try:
>>      remadd = os.environ('REMOVE_ADDR')
>>      tuple3 = socket.gethostbyaddr(remadd)
>>      host = tuple3[0]
>> except:
>>      host = type(remadd) + " : " + repr(remadd)
>>
>>
>> but iam getting an internal server error.
>>
>> I didnt print it as you said but its the same thing host var gets
>> printed later on.
>>
>> Now, why would this give an internal server error?
>>
>
> I have no idea what causes an internal server error.  It's up to you to
> get the output of the expression to some location you can examine.
> Easiest way is to run those 3 lines directly on the server, not in the
> cgi environment.
>
> But if you don't have any debugging tools, then STOP right now and build
> some.  Use logging, or redirect print, or do something that the server
> folks provide as debugging aids.  But just blindly guessing is
> ludicrous.  So also is throwing out clues by using a bare except.
>
> Assigning that string to host makes no sense at all.  And neither does
> putting it in the except clause.  You want to get that string to YOUR
> eyes, not to the server who might get an internal server error.

I don't think running it via 'cli' would help much, since its a 
cgi-script and ip addr function have no meaning calling them in plain 
our of a cgi environment but here it is:


Python 3.3.2 (default, Jun  3 2013, 16:18:05)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
 >>> import os
 >>> remadd = os.environ('REMOVE_ADDR')
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: '_Environ' object is not callable



-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/5/2013 8:49:31 AM
Στις 5/7/2013 11:27 πμ, ο/η Dave Angel έγραψε:
>
> Or if the problems cannot be recreated outside the cgi environment, use
> the log files you've been logging other problems into.  Or simply open a
> text file for writing, and add a file= keyword parameter to the print
> function call.
>
>       print(repr(e), file=myfile)

Yes you are correct, problem need to be recreated within the cgi env.

try:
	remadd = os.environ('REMOTE_ADDR')
	tuple3 = socket.gethostbyaddr(remadd)
	host = tuple3[0]
except Exception as e:
	host = repr(e)


http://superhost.gr/?show=log&page=index.html shows explicitly the same 
kind of error that python interpreter via cli gave me the same error!

-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/5/2013 9:01:18 AM
On 07/05/2013 04:49 AM, Νίκος Gr33k wrote:
>
      <SNIP>

> I don't think running it via 'cli' would help much, since its a
> cgi-script and ip addr function have no meaning calling them in plain
> our of a cgi environment but here it is:
>

No idea how to parse "have no meaning calling them in plain our of a cgi 
environment"


>
> Python 3.3.2 (default, Jun  3 2013, 16:18:05)
> [GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> import os
>  >>> remadd = os.environ('REMOVE_ADDR')
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> TypeError: '_Environ' object is not callable
>
But there were two problems with the code you faithfully copied from my 
earlier post.  One was already pointed out by feedthetroll, that I 
accidentally changed REMOTE_ADDR to REMOVE_ADDR.

The other one is perhaps more subtle; I replaced square brackets with 
parentheses.

So try again with:

  >>> import os
  >>> remadd = os.environ['REMOTE_ADDR']

I get an error:

Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "/usr/local/lib/python3.3/os.py", line 669, in __getitem__
     value = self._data[self.encodekey(key)]
KeyError: b'REMOTE_ADDR'


but presumably your machine actually has such an environment variable.

Isn't that mistake something  you could easily have caught?  Or were you 
just blindly pasting my bugs without understanding what I was trying to 
do with refactoring?

Anyway, I can't see any reason why the rest of the sequence shouldn't 
behave identically from a terminal as it does in CGI.


-- 
DaveA

0
davea9629 (629)
7/5/2013 9:21:33 AM
Στις 5/7/2013 12:21 μμ, ο/η Dave Angel έγραψε:
> On 07/05/2013 04:49 AM, Νίκος Gr33k wrote:
>>
>       <SNIP>
>
>> I don't think running it via 'cli' would help much, since its a
>> cgi-script and ip addr function have no meaning calling them in plain
>> our of a cgi environment but here it is:
>>
>
> No idea how to parse "have no meaning calling them in plain our of a cgi
> environment"
>
>
>>
>> Python 3.3.2 (default, Jun  3 2013, 16:18:05)
>> [GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux
>> Type "help", "copyright", "credits" or "license" for more information.
>>  >>> import os
>>  >>> remadd = os.environ('REMOVE_ADDR')
>> Traceback (most recent call last):
>>    File "<stdin>", line 1, in <module>
>> TypeError: '_Environ' object is not callable
>>
> But there were two problems with the code you faithfully copied from my
> earlier post.  One was already pointed out by feedthetroll, that I
> accidentally changed REMOTE_ADDR to REMOVE_ADDR.
>
> The other one is perhaps more subtle; I replaced square brackets with
> parentheses.
>
> So try again with:
>
>   >>> import os
>   >>> remadd = os.environ['REMOTE_ADDR']
>
> I get an error:
>
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>    File "/usr/local/lib/python3.3/os.py", line 669, in __getitem__
>      value = self._data[self.encodekey(key)]
> KeyError: b'REMOTE_ADDR'
>
>
> but presumably your machine actually has such an environment variable.
>
> Isn't that mistake something  you could easily have caught?  Or were you
> just blindly pasting my bugs without understanding what I was trying to
> do with refactoring?
>
> Anyway, I can't see any reason why the rest of the sequence shouldn't
> behave identically from a terminal as it does in CGI.

Yes i didnt see your typo and i have corrected it:


try:
	remadd = os.environ('REMOTE_ADDR')
	tuple3 = socket.gethostbyaddr(remadd)
	host = tuple3[0]
except Exception as e:
	host = repr(e)


Ima still receiving the same kind of erro as i did with cli as well.

You can view the error in the very first line here:

http://superhost.gr/?show=log&page=index.html


which yields: TypeError("'_Environ' object is not callable",)


-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/5/2013 9:25:29 AM
Benjamin Kaplan <benjamin.kaplan@case.edu> writes:

>> FYI, your code above is (almost) exactly equivalent to the simpler
>
> They aren't equivalent. "except Exception" won't catch KeyboardInterrupt or
> SystemExit or a few others that you really don't want to catch in a generic
> error handler. You should almost never have a bare except.

I know, that's why I added "(almost)", I was just trying to explain why
he wasn't able to see the problem.

Thanks for pointing out the difference,
ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
lele@metapensiero.it  |                 -- Fortunato Depero, 1929.

0
lele4264 (70)
7/5/2013 10:16:01 AM
Νίκος Gr33k <nikos@superhost.gr> writes:

> You can view the error in the very first line here:
>
> http://superhost.gr/?show=log&page=index.html

No, visiting that page simply emit the standard Apache error page,
without details.

>
> which yields: TypeError("'_Environ' object is not callable",)

Dave already told you the reason[1]. Νίκος, *read* **and** *understand*
our *whole* answers to your questions, otherwise we are wasting time,
you, and us!

ciao, lele.

[1] “… The other one is perhaps more subtle; I replaced square brackets
     with parentheses.”
-- 
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
lele@metapensiero.it  |                 -- Fortunato Depero, 1929.

0
lele4264 (70)
7/5/2013 10:24:54 AM
Στις 5/7/2013 12:25 μμ, ο/η Νίκος Gr33k έγραψε:
>
> try:
>      remadd = os.environ('REMOTE_ADDR')
>      tuple3 = socket.gethostbyaddr(remadd)
>      host = tuple3[0]
> except Exception as e:
>      host = repr(e)

> which yields: TypeError("'_Environ' object is not callable",)

Any thoufgs as to why os.environ('REMOTE_ADDR') gives the above error?

I noticed that if i remove my domain from cloudflare the gethostbyaddr 
as it uses too months now.
-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/5/2013 10:26:55 AM
Στις 5/7/2013 1:24 μμ, ο/η Lele Gaifax έγραψε:
> Νίκος Gr33k <nikos@superhost.gr> writes:
>
>> You can view the error in the very first line here:
>>
>> http://superhost.gr/?show=log&page=index.html
>
> No, visiting that page simply emit the standard Apache error page,
> without details.
>
>>
>> which yields: TypeError("'_Environ' object is not callable",)
>
> Dave already told you the reason[1]. Νίκος, *read* **and** *understand*
> our *whole* answers to your questions, otherwise we are wasting time,
> you, and us!
>
> ciao, lele.
>
> [1] “… The other one is perhaps more subtle; I replaced square brackets
>       with parentheses.”
>

I read carefully all of tour answer please try to load again:

http://superhost.gr/?show=log&page=index.html

Di i miss an explanation on this?

TypeError("'_Environ' object is not callable",)

-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/5/2013 10:28:45 AM
Στις 5/7/2013 12:21 μμ, ο/η Dave Angel έγραψε:
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>    File "/usr/local/lib/python3.3/os.py", line 669, in __getitem__
>      value = self._data[self.encodekey(key)]
> KeyError: b'REMOTE_ADDR


Wait!
Are you saying that the ip address is being returned as a byte string 
which then i have to decode with something like:

host = socket.gethostbyaddr( os.environ['REMOTE_HOST'].decode('utf-8') )[0]

?

-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/5/2013 10:33:05 AM
Νίκος Gr33k <nikos@superhost.gr> writes:

> Στις 5/7/2013 12:25 μμ, ο/η Νίκος Gr33k έγραψε:
>>
>> try:
>>      remadd = os.environ('REMOTE_ADDR')
>>      tuple3 = socket.gethostbyaddr(remadd)
>>      host = tuple3[0]
>> except Exception as e:
>>      host = repr(e)
>
>> which yields: TypeError("'_Environ' object is not callable",)
>
> Any thoufgs as to why os.environ('REMOTE_ADDR') gives the above error?

Yes, I'd try to understand the error message, and eventually lookup the
documentation on os.environ to see what type of object it is bound
to. The solution should be easy.

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
lele@metapensiero.it  |                 -- Fortunato Depero, 1929.

0
lele4264 (70)
7/5/2013 10:36:58 AM
Στις 5/7/2013 1:36 μμ, ο/η Lele Gaifax έγραψε:
> Νίκος Gr33k <nikos@superhost.gr> writes:
>
>> Στις 5/7/2013 12:25 μμ, ο/η Νίκος Gr33k έγραψε:
>>>
>>> try:
>>>       remadd = os.environ('REMOTE_ADDR')
>>>       tuple3 = socket.gethostbyaddr(remadd)
>>>       host = tuple3[0]
>>> except Exception as e:
>>>       host = repr(e)
>>
>>> which yields: TypeError("'_Environ' object is not callable",)
>>
>> Any thoufgs as to why os.environ('REMOTE_ADDR') gives the above error?
>
> Yes, I'd try to understand the error message, and eventually lookup the
> documentation on os.environ to see what type of object it is bound
> to. The solution should be easy.

Looks now when i print( repr(e)) i get

UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3

but what string does it try to decode and jeeps failing?


-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/5/2013 10:42:36 AM
Νίκος Gr33k <nikos@superhost.gr> writes:

> Looks now when i print( repr(e)) i get
>
> UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3
>
> but what string does it try to decode and jeeps failing?

Reasonably it's the second one, as the first clearly seems the tag of
the decoder that tried to translate it to Unicode.

As already explained, your immediate goal should be trying to understand
from *where* that byte string is coming. I can't help on that, sorry.

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
lele@metapensiero.it  |                 -- Fortunato Depero, 1929.

0
lele4264 (70)
7/5/2013 10:59:54 AM
Στις 5/7/2013 1:59 μμ, ο/η Lele Gaifax έγραψε:
> Νίκος Gr33k <nikos@superhost.gr> writes:
>
>> Looks now when i print( repr(e)) i get
>>
>> UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3
>>
>> but what string does it try to decode and jeeps failing?
>
> Reasonably it's the second one, as the first clearly seems the tag of
> the decoder that tried to translate it to Unicode.

2nd one of what? 2nd byte in order?
Can ou show me form which characters does this string consist of so we 
migth have an idea of where its coming from if we know what it looks like?

> As already explained, your immediate goal should be trying to understand
> from *where* that byte string is coming. I can't help on that, sorry.

Thats what i'm trying to do.
If i completely remove the gethostbyaddr fucntion adds function then 
there is nor problem.

The problem is recreating when the script tries to decode a hostname.

For some bizarre reason if i exclude my domain from CloudFlare then the 
reverse DNS resolution of the visitors hostname would be returned properly.

I will do it right now for you to see.
-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/5/2013 11:05:07 AM
Στις 5/7/2013 2:05 μμ, ο/η Νίκος Gr33k έγραψε:
> Thats what i'm trying to do.
> If i completely remove the gethostbyaddr fucntion adds function then
> there is nor problem.
>
> The problem is recreating when the script tries to decode a hostname.
>
> For some bizarre reason if i exclude my domain from CloudFlare then the
> reverse DNS resolution of the visitors hostname would be returned properly.
>
> I will do it right now for you to see.


Precisely as i have said it would happen:

I removed the domain form CloudFlare and your domains visting my website 
appearing as usual.

Its only when i cloudflare it and the UnicodeError happens.

But why?!?! I see no reason as to why when my domain becomes Cloudflared 
the gethostbyaddr() fails.

What kind of weird strings does it return back?


-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/5/2013 11:16:20 AM
Νίκος Gr33k <nikos@superhost.gr> writes:

> Στις 5/7/2013 1:59 μμ, ο/η Lele Gaifax έγραψε:
>> Νίκος Gr33k <nikos@superhost.gr> writes:
>>
>>> UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3
>>>
>>> but what string does it try to decode and jeeps failing?
>>
>> Reasonably it's the second one, as the first clearly seems the tag of
>> the decoder that tried to translate it to Unicode.
>
> 2nd one of what? 2nd byte in order?

You asked “what string” (although you probably meant “which string”):

UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3

first string-------^^^^^^^
second string---------------^^^^^^^^^^^^^^^^^^^^^^

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
lele@metapensiero.it  |                 -- Fortunato Depero, 1929.

0
lele4264 (70)
7/5/2013 11:16:55 AM
Στις 5/7/2013 2:16 μμ, ο/η Lele Gaifax έγραψε:
> UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3
>
> first string-------^^^^^^^
> second string---------------^^^^^^^^^^^^^^^^^^^^^^


Hold on please!

 From where do these dashes and carets characters come from?
Also from where do you see 2 strings?

Looking at that: UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3

ic an understandn onlt that utf-8 has failsed decoding some byte stream 
satrting with \xb6

I totally not follow...
-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/5/2013 11:27:25 AM
Am Freitag, 5. Juli 2013 12:33:05 UTC+2 schrieb =CE=9D=CE=AF=CE=BA=CE=BF=CF=
=82 Gr33k:
> ...
> Wait!
> Are you saying that the ip address is being returned as a byte string=20
> which then i have to decode with something like:
>=20
> host =3D socket.gethostbyaddr( os.environ['REMOTE_HOST'].decode('utf-8') =
)[0]

Wait!
I get a decode error when python tries to automatically decode a bytestring=
 assuming it to be utf-8 encoded.
I am sure the error will disappear, when I try to decode it explicit using =
utf-8. Heureka! I got it!

Or in other words:
If a big stone falls on my foot accidently it hurts.
But I am sure it will not hurt, if take that same stone and throw it on my =
foot.
Heureka! I got it!

P.S.:

Am 14.06.2013 10:35, schrieb F=C3=A1bio Santos:
> Also you have been shown this link and I feel you really need to read it.
> http://slash7.com/2006/12/22/vampires/
0
7/5/2013 11:56:47 AM
Am Freitag, 5. Juli 2013 13:27:25 UTC+2 schrieb =CE=9D=CE=AF=CE=BA=CE=BF=CF=
=82 Gr33k:
> =CE=A3=CF=84=CE=B9=CF=82 5/7/2013 2:16 =CE=BC=CE=BC, =CE=BF/=CE=B7 Lele G=
aifax =CE=AD=CE=B3=CF=81=CE=B1=CF=88=CE=B5:
>> UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3
>> first string-------^^^^^^^
>> second string---------------^^^^^^^^^^^^^^^^^^^^^^
>=20
> Hold on please!
>  From where do these dashes and carets characters come from?

ROTFL!!!!
Nikos, you made my day, again! Fun is back in these threads!

>=20
> Also from where do you see 2 strings?

Look, my little dumb baby: The dashes and carets point to the strings.
The first one being 'utf-8', the second one starting with '\xb6\xe3\xed' (b=
eing a bytestring, therefore the b before the ')
Sorry, I forgot. You are not using python for your business, therefore you =
can't know, that strings in python can for example be identified by surroun=
ding '.

> ic an understandn onlt that utf-8 has failsed decoding some byte stream=
=20
> satrting with \xb6
So ... where did you call utf-8() so that it could try to decode something?
0
7/5/2013 12:06:17 PM
Am Freitag, 5. Juli 2013 12:33:05 UTC+2 schrieb =CE=9D=CE=AF=CE=BA=CE=BF=CF=
=82 Gr33k:
> ...
> Wait!
> Are you saying that the ip address is being returned as a byte string=20
> which then i have to decode with something like:
>=20
> host =3D socket.gethostbyaddr( os.environ['REMOTE_HOST'].decode('utf-8') =
)[0]

Wait!
I get a decode error when python tries to automatically decode a bytestring
assuming it to be utf-8 encoded.
I am sure the error will disappear, when I try to decode it explicit using
utf-8. Heureka! I got it!

Or in other words:
If a big stone falls on my foot accidently, it hurts.
------------------------------------------^
But I am sure it will not hurt, if take that same stone and throw it on my =
foot.

Heureka! I got it!



P.S.:

Am 14.06.2013 10:35, schrieb F=C3=A1bio Santos:
> Also you have been shown this link and I feel you really need to read it.
> http://slash7.com/2006/12/22/vampires/
0
7/5/2013 12:14:13 PM
On 07/05/2013 06:33 AM, Νίκος Gr33k wrote:
> Στις 5/7/2013 12:21 μμ, ο/η Dave Angel έγραψε:
>> Traceback (most recent call last):
>>    File "<stdin>", line 1, in <module>
>>    File "/usr/local/lib/python3.3/os.py", line 669, in __getitem__
>>      value = self._data[self.encodekey(key)]
>> KeyError: b'REMOTE_ADDR
>
>
> Wait!
> Are you saying that the ip address is being returned as a byte string
> which then i have to decode with something like:
>
> host = socket.gethostbyaddr( os.environ['REMOTE_HOST'].decode('utf-8') )[0]
>

Don't fix the problem till you understand it.  Figure out who is dealing 
with a byte string here, and where that byte string came from.  Adding a 
decode, especially one that's going to do the same decode as your 
original error message, is very premature.

You're quoting from my error output, and that's caused because I don't 
have such an environment variable.  But you do.  So why aren't you in 
there debugging it?  And why on earth are you using the complex 
expression instead of a refactored one which might be simple enough for 
you to figure out what's wrong with it.

There is definitely something strange going on with that os.environ 
reference (NOT call). So have you yet succeeded in running the factored 
lines?  If you can't get them to run, at least up to the point that you 
get that unicode error, then you'll make progress only by guessing.

Get to that interactive debug session, and enter the lines till you get 
an error.  Then at least you know which line is causing the error.

    xxx = os.environ['REMOTE_HOST']
    yyy = socket.gethostbyaddr(xxx)
    host = yyy[0]


I'll bet the real problem is you're using some greek characters in the 
name of the environment variable, rather than "REMOTE_HOST"  So 
everything you show us is laboriously retyped, hiding the real problems 
underneath.


-- 
DaveA

0
davea9629 (629)
7/5/2013 1:05:33 PM
Dave Angel <davea@davea.name> writes:

> You're quoting from my error output, and that's caused because I don't
> have such an environment variable.  But you do.

Dave, maybe you already know, but that variable is "injected" by the CGI
mechanism, is not coming from the OP shell environment.

As Νίκος discovered, when he "cloudfare" (whatever that means) his site,
the REMOTE_HOST envvar contains some (I guess) latin-greek encoded
string, and the remote address is carried by a different envvar...

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
lele@metapensiero.it  |                 -- Fortunato Depero, 1929.

0
lele4264 (70)
7/5/2013 2:11:20 PM
Στις 5/7/2013 5:11 μμ, ο/η Lele Gaifax έγραψε:
> Dave Angel <davea@davea.name> writes:
>
>> You're quoting from my error output, and that's caused because I don't
>> have such an environment variable.  But you do.
>
> Dave, maybe you already know, but that variable is "injected" by the CGI
> mechanism, is not coming from the OP shell environment.
>
> As Νίκος discovered, when he "cloudfare" (whatever that means) his site,
> the REMOTE_HOST envvar contains some (I guess) latin-greek encoded
> string, and the remote address is carried by a different envvar...

Exactly only when i CloudFlare(www.cloudflare.com) the domain the 
hostname cannot be retrieved.

At least i managed to solve this by:

try:
   host = socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] )[0]
except Exception as e:
   host = repr(e)


Seems like when you cloudflare a domain you can  no longer have the
originates ip address of the visitor but you have to read the above
environmental variable to be bale to retrieve it!

-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/5/2013 2:27:31 PM
  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--8323329-1415386919-1373629673=:2674
Content-Type: TEXT/PLAIN; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8BIT

On Thu, 4 Jul 2013, Νίκος Γκρ33κ wrote:

> Στις 4/7/2013 6:10 μμ, ο/η MRAB έγραψε:
>> What do you mean "I don't know how to catch the exception with
>> OSError"? You've tried "except socket.gaierror" and "except
>> socket.herror", well just write "except OSError" instead!
>
>
> try:
> 	host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
> except OSError:
> 	host = "UnResolved"
>
> produces also an internal server error.
>
> Are you sure is just except OSError ?
>

Have you ensured that 'REMOTE_ADDR' is actually a key in os.environ? I 
highly recommend using the logging module to help diagnose what the actual 
exception is.

HTH,
-W
--8323329-1415386919-1373629673=:2674--
0
wayne9977 (50)
7/12/2013 11:47:53 AM
Στις 12/7/2013 2:47 μμ, ο/η Wayne Werner έγραψε:
> On Thu, 4 Jul 2013, Νίκος Γκρ33κ wrote:
>
>> Στις 4/7/2013 6:10 μμ, ο/η MRAB έγραψε:
>>> What do you mean "I don't know how to catch the exception with
>>> OSError"? You've tried "except socket.gaierror" and "except
>>> socket.herror", well just write "except OSError" instead!
>>
>>
>> try:
>>     host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
>> except OSError:
>>     host = "UnResolved"
>>
>> produces also an internal server error.
>>
>> Are you sure is just except OSError ?
>>
>
> Have you ensured that 'REMOTE_ADDR' is actually a key in os.environ? I
> highly recommend using the logging module to help diagnose what the
> actual exception is.
>
> HTH,
> -W

Yes it is a key, but the problem as i suspected was cloudflare.
i had to use os.environ['HTTP_CF_CONNECTING_IP'] that cloudflare passes 
as variable i the cgi enviroment in order to retrieve the visitor's ip.


try:
	gi = pygeoip.GeoIP('/usr/local/share/GeoLiteCity.dat')
	city = gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] )
	host = socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] )[0]
except Exception as e:
	host = repr(e)


Sometimes though iam still receiving the usual
UnicodeDecodeError('utf-8', b'\xc1\xf0\xef\xf4\xf5

but only for a few ip addresses, in moste cases it works.
-- 
What is now proved was at first only imagined!
0
nikos6878 (207)
7/12/2013 11:56:07 AM
On 07/12/2013 07:56 AM, Ferrous Cranus wrote:
> Στις 12/7/2013 2:47 μμ, ο/η Wayne Werner έγραψε:
>> On Thu, 4 Jul 2013, Νίκος Γκρ33κ wrote:
>>
>>> Στις 4/7/2013 6:10 μμ, ο/η MRAB έγραψε:
>>>> What do you mean "I don't know how to catch the exception with
>>>> OSError"? You've tried "except socket.gaierror" and "except
>>>> socket.herror", well just write "except OSError" instead!
>>>
>>>
>>> try:
>>>     host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0]
>>> except OSError:
>>>     host = "UnResolved"
>>>
>>> produces also an internal server error.
>>>
>>> Are you sure is just except OSError ?
>>>
>>
>> Have you ensured that 'REMOTE_ADDR' is actually a key in os.environ? I
>> highly recommend using the logging module to help diagnose what the
>> actual exception is.
>>
>> HTH,
>> -W
>
> Yes it is a key, but the problem as i suspected was cloudflare.
> i had to use os.environ['HTTP_CF_CONNECTING_IP'] that cloudflare passes
> as variable i the cgi enviroment in order to retrieve the visitor's ip.
>
>
> try:
>      gi = pygeoip.GeoIP('/usr/local/share/GeoLiteCity.dat')
>      city = gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] )
>      host = socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] )[0]
> except Exception as e:
>      host = repr(e)
>
>
> Sometimes though iam still receiving the usual
> UnicodeDecodeError('utf-8', b'\xc1\xf0\xef\xf4\xf5
>
> but only for a few ip addresses, in moste cases it works.

And naturally, you now know how to debug those UnicodeDecodeError 
problems.  Surely, the code you post here isn't what you actually do, 
because when people spend time to give you detailed advice, you actually 
read it, and work at understanding it.

Chortle, snort.

-- 
DaveA

0
davea9629 (629)
7/12/2013 3:46:35 PM
shut off the HD, start the SO form an USB and shut on again the HD

0
roggero.n (1)
9/6/2013 11:17:58 PM
Reply:

Similar Artilces:

[pysqlite 2.0.2] UnicodeDecodeError: 'utf8' codec can't decode bytes in position 44-45: invalid data
A fetchall() call causes the above error. There must be some non-utf8 chars in the table in question and - indeed - there very likely are: The table contains pathnames of files on the harddisk of a German "speaking" PC. What I have already done to try to overcome this is: - Inserted "# coding: iso-8859-1" (w/o the quotes) in (hopefully) every file of my project - All strings I could find prefixed with "u" (w/o the quotes) - Changed plain calls to execute(q) into execute(unicode(q)) Alas, I am not sure if this all makes really sense, and, obviously it doesn't...

RE: UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)
This is a multipart message in MIME format. --Boundary_(ID_oK5MBHtGHXoVbM6v7o3uQQ) Content-type: text/plain; charset=utf-8 Content-transfer-encoding: quoted-printable Hi Terry, Thanks for your comment. =20 >To better help us help you, what exact version of Python? I use Python 2.7.5 =20 >Please post plain text without html. Sorry, I will do it. =20 >Please post programs single spaced with just occasional blank lines. =20 File my_program.py =20 import wx.aui import paramiko import telnetlib from LinuxHostPage import * from WindowsHostPage import * from AIXHostPage import * Publisher =3D pub.Publisher() TIMEOUT =3D 3 =20 class MainWindow(wx.Frame): def __init__(self, parent, title): =20 self.os =3D '' self.connection =3D [] =20 wx.Frame.__init__(self, parent, title=3Dtitle) self.SetBackgroundColour('LightGray') =20 # A Statusbar in the bottom of the window self.CreateStatusBar()=20 self.SetStatusText("Please input credentials.") =20 # Publisher Publisher().subscribe(self.change_statusbar, 'change_statusbar') =20 # Input box self.input_panel =3D wx.Panel(self) self.host =3D wx.StaticText(self.input_panel, -1, 'Host') self.host_entry =3D wx.TextCtrl(self.input_...

UnicodeDecodeError: 'ascii' codec can't decode byte 0xa0 in position 10: ordinal not in range(128)
I have no idea what is causing this error, or how to fix it. The full error is: Traceback (most recent call last): File "D:\ScriptRuntime\PS\Automation\Handlers\SCMTestToolResourceToolsBAT.py", line 60, in Run PS.Automation.Utility.System.AppendSystemPath(args["PATH"], context) File "D:\ScriptRuntime\PS\Automation\Utility\System.py", line 55, in AppendSys temPath AppendPathVariable("PATH", appendtext, context) File "D:\ScriptRuntime\PS\Automation\Utility\System.py", line 37, in AppendPat hVariable if(ap == pp): UnicodeDecod...

UnicodeDecodeError: 'ascii' codec can't decodee byte 0xff in position 0: ordinal not in range(128)
This is a multipart message in MIME format. --Boundary_(ID_E/4IbDwDEnubOe6Ijgqz/A) Content-type: text/plain; charset=utf-8 Content-transfer-encoding: quoted-printable Hi all, =20 I use Python telnetlib on Windows 7 32 bit. Here is my code: =20 def telnet(self, host, os, username, password): connection =3D telnetlib.Telnet(host) connection.read_until('login: ') connection.write(username + '\r') connection.read_until('assword: ') connection.write(password + '\r') connection...

UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 308: character maps to <undefined>
For 'mimetypes' in the code given below, python is giving the following error. Kindly help. >>> import os >>> matches = [] >>> for (dirname, dirshere, fileshere) in os.walk(r'C:\Python34'): for filename in fileshere: if filename.endswith('.py'): pathname = os.path.join(dirname, filename) if 'mimetypes' in open(pathname).read(): matches.append(pathname) Traceback (most recent call last): File "<pyshell#165>", line 5, in <module> if 'mimetypes' in open(pathname).read()...

UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 10442: character maps to <undefined>
Im reading a file. But there seems to be some encoding error. >>> f =3D open(filename) >>> data =3D f.read() Traceback (most recent call last): File "<pyshell#2>", line 1, in <module> data =3D f.read() File "C:\Python30\lib\io.py", line 1724, in read decoder.decode(self.buffer.read(), final=3DTrue)) File "C:\Python30\lib\io.py", line 1295, in decode output =3D self.decoder.decode(input, final=3Dfinal) File "C:\Python30\lib\encodings\cp1252.py", line 23, in decode return codecs.charmap_decode(input,se...

UnicodeDecodeError: 'ascii' codec can't decode byte
Hello It seems like I have Unicode data in a CSV file but Python is using a different code page, so isn't happy when I'm trying to read and put this data into an SQLite database with APSW: ======== sql = "INSERT INTO mytable (col1,col2) VALUES (?,?)" cursor.executemany(sql, records("test.tsv")) """ UnicodeDecodeError: 'ascii' codec can't decode byte 0xc9 in position 18: ordinal not in range(128) """ ======== What should I do so Python doesn't raise this error? Should I convert data in the CVS file, or is there some f...

Long way around UnicodeDecodeError, or 'ascii' codec can't decode byte
Hello, I'm working on an unicode-aware application. I like to use "print" to debug programs, but in this case it was nightmare. The most popular result of "print" was: UnicodeDecodeError: 'ascii' codec can't decode byte 0xXX in position 0: ordinal not in range(128) I spent two hours fixing it, and I hope it's done. The solution is one of the ugliest hack I ever written, but it solves the pain. The full story and the code is in my blog: http://uucode.com/blog/2007/03/23/shut-up-you-dummy-7-bit-python/ -- Oleg Parashchenko olpa@ http://uucode.com/ ht...

[2.5.1] "UnicodeDecodeError: 'ascii' codec can't decode byte"?
Hello I'm getting this error while downloading and parsing web pages: ===== title = m.group(1) UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 48: ordinal not in range(128) ===== From what I understand, it's because some strings are Unicode, and hence contain characters that are illegal in ASCII. Does someone know how to solve this error? Thank you. Gilles Ganault wrote: > I'm getting this error while downloading and parsing web pages: > > ===== > title = m.group(1) > UnicodeDecodeError: 'ascii' codec can't...

SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape
The script is very simple (abc.txt exists in ROOTDIR directory): import os import shutil ROOTDIR = 'C:\Users\zoran' file1 = os.path.join(ROOTDIR, 'abc.txt') file2 = os.path.join(ROOTDIR, 'def.txt') shutil.move(file1, file2) But it returns the following error: C:\Python34\python.exe C:/Users/bckslash_test.py File "C:/Users/bckslash_test.py", line 4 ROOTDIR = 'C:\Users' ^ SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape Process ...

UnicodeEncodeError: 'ascii' codec can't encode character u'\ua000' in position 0: ordinal not in range(128)
Hi, I am trying to understand what does encode() do. What are the hex representations of "u" in main.py? Why there is UnicodeEncodeError when main.py is piped to xxd? Why there is no such error when it is not piped? Thanks. ~$ cat main.py #!/usr/bin/env python u =3D unichr(40960) + u'abcd' + unichr(1972) print u ~$ cat main_encode.py #!/usr/bin/env python u =3D unichr(40960) + u'abcd' + unichr(1972) print u.encode('utf-8') $ ./main.py =EA=80=80abcd=DE=B4 ~$ cat main.sh #!/usr/bin/env bash set -v ../main.py | xxd ../main_encode.py |...

'ascii' codec can't encode character u'\xe4' in position 4: ordinal not in range(128)
Hello friends, I am using pyWPS + GRASS to generate the maps for the given request XML. As my requestxml contains scandinavian letters , i got the following error, " 'ascii' codec can't encode character u'\xe4' in position 4: ordinal not in range(128) " The Request xml also contains encoding specification like <?xml version="1.0" encoding="UTF-8" standalone="yes"?> Then i traced the pywps code and added the following line, inputXml = inputXml.encode("utf-8") But it gives ...

UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 13: ordinal not in range(128)
Hi! Can anyone please help me getting rid of this error: UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 13: ordinal not in range(128) I am not a python programmer (though intend to start learning this wonderful language), I am just using a python script. After doing some search, I found that 0xb7 is a 'middle dot character' that is not interpreted by the python. Even after inserting text = text.replace('\u00b7', '') in the script, the problem still persists. Can anyone please tell me the easiest way to get rid of th...

UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 99: ordinal not in range(128)
Hi, I don't know what I'm doing wrong here. I''m using Python 2.4 and py2exe. I get he following error: Traceback (most recent call last): File "notegui.pyc", line 34, in OnClose File "brain.pyc", line 61, in setNote File "points.pyc", line 151, in setNote File "point.pyc", line 100, in writeNote UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in position 99: ordinal not in range(128) The piece of code involved is: noteFileObj = open(noteFile, "wb") noteFileObj.write(...

'ascii' codec can't encode character u'\u2013'
Hi Using Python 2.3.4 + Feedparser 3.3 (a library to parse XML documents) I'm trying to parse a UTF-8 document with special characters like acute-accent vowels: -------- <?xml version=3D"1.0" encoding=3D"UTF-8" standalone=3D"yes"?> .... ------- But I get this error message: ------- UnicodeEncodeError: 'ascii' codec can't encode character u'\u2013' in position 122: ordinal not in range(128) ------- when trying to execute a MySQL query: ---- query =3D "UPDATE blogs_news SET text =3D '" + text_extrated + "'WHER...

'ascii' codec can't encode character u'\xf3'
Hi, I get a piece of code of ogg123.py from the pyogg site, this is the code: ******************************* ogg_file = sys.argv[1] vorbis_file=ogg.vorbis.VorbisFile(ogg_file) comentarios = vorbis_file.comment() recognized_comments = ('Artist', 'Album', 'Title', 'Version', 'Organization', 'Genre', 'Description', 'Date', 'Location', 'Copyright', 'Vendor') comment_dict = {} for com in recognized_comments: comment_dict[string.upper(com)] = '%s: %%s' % com known_keys = comment_dict.keys(...

'is not' or '!='
A newbie question to you; what is the difference between statements like: if x is not None: and if x != None: Without any context, which one should be preferred? IMHO, the latter is more readable. On 2014-08-18 21:35, ElChino wrote: > A newbie question to you; what is the difference between statements > like: > if x is not None: > and > if x != None: > > Without any context, which one should be preferred? > IMHO, the latter is more readable. > "x == y" tells you whether x and y refer to objects that are equal. "x is y" tells you whether x and y actually refer to the same object. In the case of singletons like None (there's only one None object), it's better to use "is". "ElChino" <elchino@cnn.cn>: > A newbie question to you; what is the difference between statements > like: > if x is not None: > and > if x != None: Do the following: take two $10 bills. Hold one bill in the left hand, hold the other bill in the right hand. Now, the bill in the left hand "is not" the bill in the right hand. However, the bill in the left hand "==" the bill in the right hand. > Without any context, which one should be preferred? > IMHO, the latter is more readable. In almost all cases, both tests would result in the same behavior. However, the "is not" test is conceptually the correct one since you want...

'^=' and '~='?
Hello, What is the difference between '^=' and '~='? Thanks, Duckhye ...

lynx 'browser' can't do "<?xml version="1.0" encoding="utf-8"?>"
lynx is lighteningly fast to fetch a set of pages [via a script] via dump > fileID. But [my current version] doesn't render the 'style' of: "<?xml version="1.0" encoding="utf-8"?>" What is this new "xml" crap ? Will it likely take over the older *.html ? Thanks for any info, == Chris Glur. news@absamail.co.za wrote in news:1172596265.663611@vasbyt.isdsl.net: > lynx is lighteningly fast to fetch a set of pages [via a script] via > dump > fileID. > > But [my current version] doesn't ...

Fun with 'str' and 'bytes'
Hi all I want to create a cookie containing a session id. In python 2.6 I had the following - from __future__ import unicode_literals session_id = b64encode(urandom(20)) response_headers.append( (b'Set-Cookie', b'sid="{0}"'.format(session_id))) After upgrading to 3.2, the above lines generate this traceback - AttributeError: 'bytes' object has no attribute 'format' The best workaround I can come up with is the following - session_id = b64encode(urandom(20)) response_headers.append( (b'Set-Cookie', b'sid="' + sessi...

Why can't we 'tformfwd' a cp2tform(..., 'polynomial')
Hi, Just as an example: This is allowed xybase = reshape(randn(12,1),6,2); t_affine = cp2tform(xybase,xybase,'affine'); [x,y] = tformfwd(t_affine,xybase,xybase); but this t_poly = cp2tform(xybase,xybase,'polynomial',2); [x,y] = tformfwd(t_poly,xybase,xybase); gives an error "Function TFORMFWD expected T's forward_fcn field to be non- empty." I checked the code a bit and indeed, with the 'polynomial' case, the findPolynomialTransform() function has at its end trans = maketform('custom',2,2,[],@inv_polynomial,[A B]); wher...

python3; ftplib: TypeError: Can't convert 'bytes' object to str implicitly
This is the code I run (python 3.3) host = ... user = ... passwd = ... from ftplib import FTP ftp = FTP(host, user, passwd) ftp.mkd(b'NewDir') ftp.rmd(b'NewDir') This is the traceback Traceback (most recent call last): File "ftp-problem", line 9, in <module> ftp.mkd(b'NewDir') File "/usr/lib/python3.3/ftplib.py", line 612, in mkd resp = self.voidcmd('MKD ' + dirname) TypeError: Can't convert 'bytes' object to str implicitly The problem is that I do something like this in a backup progra...

Removing the letters 't' 'i' 'x' and 'y'
Im having trouble with figuring out how to write a function which will remove the letters 't' 'i' 'x' and 'y' from any input string in matlab. for example modstr(pixy) should result in ans = 'p' or modstr(picture) should result in ans = 'pcure' if anyone could offer any help or hints i would really appreciate it. Thank You >> modstr=@(s) s(~ismember(s,'tixy')) modstr = @(s)s(~ismember(s,'tixy')) >> modstr('picture') ans = pcure % Bruno ...

How come I can't use 'sqrt' and 'fabs'?
Hi, group, When I compile my code, the error message are: "error C2668: 'fabs' : ambiguous call to overloaded function" "error C2668: 'sqrt' : ambiguous call to overloaded function" I already add the math library, what does this error mean? Thanks. Vol. Vol wrote: > Hi, group, > When I compile my code, the error message are: > "error C2668: 'fabs' : ambiguous call to overloaded function" > "error C2668: 'sqrt' : ambiguous call to overloaded function" > > I already add the math library, what does this...

Web resources about - UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb6 in position 0: invalid start byte - comp.lang.python

Resources last updated: 3/4/2016 7:11:22 PM