I ran into a problem with capital accented letters in Biblatex 0.9
running under lua(la)tex.

The enclosed sample file and sample bib compile fine in pdflatex and
xetex, but stop working under lualatex. The error I receive is:

! String contains an invalid utf-8 sequence.
../sample.bbl
l.23      {=C3=89mile}{
                   =EF=BF=BD.}%

and has to do with the capital accented E in the author's first name.
Luatex does not seem to have any problem with it---the same character
in the body text works fine. But luatex somehow gets in  gets in
trouble when dealing with the .bbl file created by biblatex, whereas
xetex deals with it just fine (the .bbl files are identical). Or so it
seems to me. I have no idea who is it to blame, luatex or biblatex,
and I would appreciate any hint toward a solution.

Cheers,

Stefano

-----minimal example-----

%%%%% sample.tex file

\documentclass[letterpaper,10pt]{memoir}
\usepackage{fontspec}
\setromanfont[Mapping=3Dtex-text]{TeX Gyre Termes}
\setsansfont[Mapping=3Dtex-text]{DejaVu Sans}
\setmonofont[Mapping=3Dtex-text]{DejaVu Sans Mono}
\usepackage[natbib=3Dtrue, indexing=3Dcite, citestyle=3Dauthoryear-ibid,
bibstyle=3Dauthoryear, sorting=3Dnyt, indexing=3Dcite, abbreviate=3Dtrue,
useprefix=3Dfalse]{biblatex}
\bibliography{sample.bib}

\begin{document}
Sample text with one citation:
 \cite{durkheim1912}

=C3=89mile Durkheim
\printbibliography

\end{document}


%%%%%% sample.bib file

@BOOK{Durkheim1912,
  title =3D {Les formes =C3=A9l=C3=A9mentaires de la vie religieuse. Le sys=
t=C3=A8me
tot=C3=A9mique en Australie},
  publisher =3D {Presses Universitaires de France},
  year =3D {1912},
  author =3D {=C3=89mile Durkheim},
  address =3D {Paris},
  language =3D {french},
   timestamp =3D {2010.04.05}
}

On Nov 12, 10:14=A0pm, stefano <stefano.fran...@gmail.com> wrote:
> I ran into a problem with capital accented letters in Biblatex 0.9
> running under lua(la)tex.
>
> The enclosed sample file and sample bib compile fine in pdflatex and
> xetex, but stop working under lualatex. The error I receive is:
>
> ! String contains an invalid utf-8 sequence.
> ./sample.bbl
> l.23 =A0 =A0 =A0{=C9mile}{
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 .}%
>
> and has to do with the capital accented E in the author's first name.
> Luatex does not seem to have any problem with it---the same character
> in the body text works fine. But luatex somehow gets in =A0gets in
> trouble when dealing with the .bbl file created by biblatex, whereas
> xetex deals with it just fine (the .bbl files are identical). Or so it
> seems to me. I have no idea who is it to blame, luatex or biblatex,
> and I would appreciate any hint toward a solution.

How do you generate the .bbl file? If I run lualatex, bibtexu (notice
the u), then lualatex twice, I get no errors and the pdf seems fine,
however if i replace bibtexu with bibtex I get the same error above
which is expected since bibtex is not Unicode enabled, bibtexu is
Unicode enabled version of bibtex (AFAICT) but I guess you can used
biber too. I'm not sure how does it work for xetex, though.

stefano wrote:

> I ran into a problem with capital accented letters in Biblatex 0.9
> running under lua(la)tex.
> 
> The enclosed sample file and sample bib compile fine in pdflatex and
> xetex, but stop working under lualatex. The error I receive is:
> 
> ! String contains an invalid utf-8 sequence.

If your bib file in UTF8-encoded, you need an UTF8-savvy backend.
Set backend=biber and get Biber from:

https://sourceforge.net/projects/biblatex-biber/files/biblatex-
biber/current/binaries/

See §2.4.3 in the biblatex manual for further explanation.

-- 
Sender address blackholed, do not reply directly.
You can still reach me by email at: lehman gmx net.

Am Fri, 12 Nov 2010 12:14:37 -0800 (PST) schrieb stefano:

> I ran into a problem with capital accented letters in Biblatex 0.9
> running under lua(la)tex.
> 
> The enclosed sample file and sample bib compile fine in pdflatex and
> xetex, but stop working under lualatex. The error I receive is:
> 
> ! String contains an invalid utf-8 sequence.
> ./sample.bbl
> l.23      {Émile}{
>                    �.}%

Xetex is probably complaining too but not so loud. When xetex
encounters an invalid utf-8 sequence it writes a message in the log:

Invalid UTF-8 byte or sequence at line 23 replaced by U+FFFD.


The message means either that your bib isn't utf8 encoded or that
bibtex (which is an 8-bit-application) damaged an utf8-char.  


-- 
Ulrike Fischer 

On Nov 13, 4:16=A0am, Ulrike Fischer <ne...@nililand.de> wrote:
> Am Fri, 12 Nov 2010 12:14:37 -0800 (PST) schrieb stefano:
>
> > I ran into a problem with capital accented letters in Biblatex 0.9
> > running under lua(la)tex.
>
> > The enclosed sample file and sample bib compile fine in pdflatex and
> > xetex, but stop working under lualatex. The error I receive is:
>
> > ! String contains an invalid utf-8 sequence.
> > ./sample.bbl
> > l.23 =A0 =A0 =A0{=C9mile}{
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 .}%
>
> Xetex is probably complaining too but not so loud. When xetex
> encounters an invalid utf-8 sequence it writes a message in the log:
>
> Invalid UTF-8 byte or sequence at line 23 replaced by U+FFFD.
>
> The message means either that your bib isn't utf8 encoded or that
> bibtex (which is an 8-bit-application) damaged an utf8-char. =A0
>
> --
> Ulrike Fischer

Thanks to Ulrike, Khaled, and Philipp for their feedback.I think I
understand the problem better. Unfortunately I am still stuck.
because:

1. bibtexu does indeed compile correctly the test file I posted. But
when I try it on my real file (which has about 700 refs) it gets stuck
into an infinite loop and eventually eats up all my memory until it
crashes my workstation.

2. I cannot get the current version of  biber to work (0.5.7). The
binary file distributed by the biber project fails to process my ref
file complaining about all sorts of missing perl modules. I tried  to
install from source, but ran into similar dependency-related problems
(I posted a query on the biber forums about it).

So, it's back to XeTeX for now. Very disappointng.

Stefano

stefano <stefano.franchi@gmail.com> writes:

> 2. I cannot get the current version of  biber to work (0.5.7). The
> binary file distributed by the biber project fails to process my ref
> file complaining about all sorts of missing perl modules. I tried  to
> install from source, but ran into similar dependency-related problems
> (I posted a query on the biber forums about it).

The binaries are intended to be self-contained, so please report that at
http://sourceforge.net/tracker/?group_id=228270&atid=1073792.

-- 
Change “LookInSig” to “tcalveu” to answer by mail.

On 11/13/2010 07:54 PM, Philipp Stephani wrote:
> stefano<stefano.franchi@gmail.com>  writes:
>
>> 2. I cannot get the current version of  biber to work (0.5.7). The
>> binary file distributed by the biber project fails to process my ref
>> file complaining about all sorts of missing perl modules. I tried  to
>> install from source, but ran into similar dependency-related problems
>> (I posted a query on the biber forums about it).

With biber you will have a much better chance of your problems
actually being looked at than with bibtexu, which appears to be
abandonware.

Good luck,
Taco

On Nov 13, 8:51=A0pm, stefano <stefano.fran...@gmail.com> wrote:
> On Nov 13, 4:16=A0am, Ulrike Fischer <ne...@nililand.de> wrote:
>
>
>
> > Am Fri, 12 Nov 2010 12:14:37 -0800 (PST) schrieb stefano:
>
> > > I ran into a problem with capital accented letters in Biblatex 0.9
> > > running under lua(la)tex.
>
> > > The enclosed sample file and sample bib compile fine in pdflatex and
> > > xetex, but stop working under lualatex. The error I receive is:
>
> > > ! String contains an invalid utf-8 sequence.
> > > ./sample.bbl
> > > l.23 =A0 =A0 =A0{=C9mile}{
> > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 .}%
>
> > Xetex is probably complaining too but not so loud. When xetex
> > encounters an invalid utf-8 sequence it writes a message in the log:
>
> > Invalid UTF-8 byte or sequence at line 23 replaced by U+FFFD.
>
> > The message means either that your bib isn't utf8 encoded or that
> > bibtex (which is an 8-bit-application) damaged an utf8-char. =A0
>
> > --
> > Ulrike Fischer
>
> Thanks to Ulrike, Khaled, and Philipp for their feedback.I think I
> understand the problem better. Unfortunately I am still stuck.
> because:
>
> 1. bibtexu does indeed compile correctly the test file I posted. But
> when I try it on my real file (which has about 700 refs) it gets stuck
> into an infinite loop and eventually eats up all my memory until it
> crashes my workstation.
>
> 2. I cannot get the current version of =A0biber to work (0.5.7). The
> binary file distributed by the biber project fails to process my ref
> file complaining about all sorts of missing perl modules. I tried =A0to
> install from source, but ran into similar dependency-related problems
> (I posted a query on the biber forums about it).
>
> So, it's back to XeTeX for now. Very disappointng.

Note that even with XeTeX you still have a problem, the file may
compile but are you sure the text is OK in the PDF file?

As for biber, I just installed it from tlcontrib repository:

 tlmgr --repository http://tlcontrib.metatex.org/2010 install biber

It seems to run but I've no real test files to actually check.

Regards,
 Khaled

On Nov 13, 1:17=C2=A0pm, Taco Hoekwater <t...@elvenkind.com> wrote:
> On 11/13/2010 07:54 PM, Philipp Stephani wrote:
>
> > stefano<stefano.fran...@gmail.com> =C2=A0writes:
>
> >> 2. I cannot get the current version of =C2=A0biber to work (0.5.7). Th=
e
> >> binary file distributed by the biber project fails to process my ref
> >> file complaining about all sorts of missing perl modules. I tried =C2=
=A0to
> >> install from source, but ran into similar dependency-related problems
> >> (I posted a query on the biber forums about it).
>
> With biber you will have a much better chance of your problems
> actually being looked at than with bibtexu, which appears to be
> abandonware.
>
> Good luck,
> Taco

Update:

After the failed installation of biber from source (but almost
completely successful installation of perl modules), I retried
installing the self-contained biber binary. Now it runs to completion
when processing the sample file I posted earlier. Unfortunately, it
makes things worse. Now all accented characters are corrupted. See
below for the .bbl file.
More and more puzzled.

Stefano


bbl file:

% $ biblatex auxiliary file $
% $ biblatex version 0.9e $
% $ biber version 0.6 (beta)$
% Do not modify the above lines!
%
% This is an auxiliary file used by the 'biblatex' package.
% This file may safely be deleted. It will be recreated by
% biber or bibtex as required.
%
\begingroup
\makeatletter
\@ifundefined{ver@biblatex.sty}
  {\@latex@error
     {Missing 'biblatex' package}
     {The bibliography requires the 'biblatex' package.}
      \aftergroup\endinput}
  {}
\endgroup


\refsection{0}
  \entry{durkheim1912}{book}{}
    \name{author}{1}{%
      {{Durkheim}{D.}{=EF=BF=BD=EF=BF=BDmile}{=EF=BF=BD.}{}{}{}{}}%
    }
    \list{publisher}{1}{%
      {Presses Universitaires de France}%
    }
    \list{location}{1}{%
      {Paris}%
    }
    \list{language}{1}{%
      {french}%
    }
    \strng{namehash}{D=EF=BF=BD1}
    \strng{fullhash}{D=EF=BF=BD1}
    \field{sortinit}{D}
    \field{labelyear}{1912}
    \count{uniquename}{0}
    \field{year}{1912}
    \field{title}{Les formes =EF=BF=BD=EF=BF=BDl=EF=BF=BD=EF=BF=BDmentaires=
 de la vie religieuse. Le
syst=EF=BF=BD=EF=BF=BDme tot=EF=BF=BD=EF=BF=BDmique en Australie}
  \endentry

\endrefsection
\endinput

On Nov 13, 8:36=A0pm, stefano <stefano.fran...@gmail.com> wrote:
> On Nov 13, 1:17=A0pm, Taco Hoekwater <t...@elvenkind.com> wrote:
>
>
>
>
>
>
>
>
>
> > On 11/13/2010 07:54 PM, Philipp Stephani wrote:
>
> > > stefano<stefano.fran...@gmail.com> =A0writes:
>
> > >> 2. I cannot get the current version of =A0biber to work (0.5.7). The
> > >> binary file distributed by the biber project fails to process my ref
> > >> file complaining about all sorts of missing perl modules. I tried =
=A0to
> > >> install from source, but ran into similar dependency-related problem=
s
> > >> (I posted a query on the biber forums about it).
>
> > With biber you will have a much better chance of your problems
> > actually being looked at than with bibtexu, which appears to be
> > abandonware.
>
> > Good luck,
> > Taco
>
> Update:
>
> After the failed installation of biber from source (but almost
> completely successful installation of perl modules), I retried
> installing the self-contained biber binary. Now it runs to completion
> when processing the sample file I posted earlier. Unfortunately, it
> makes things worse. Now all accented characters are corrupted. See
> below for the .bbl file.
> More and more puzzled.

I replied to this on the SF forum for Biber but the solution is that
you'll need to add the "bibencoding=3Dutf8" option for biblatex since
your .bib is UTF-8. It defaults to ascii otherwise and so biber tries
to re-encode an ascii .bib to a UTF-8 .bbl which isn't going to work
when your .bib is really UTF-8 ...

On Nov 13, 1:36=A0pm, stefano <stefano.fran...@gmail.com> wrote:
> On Nov 13, 1:17=A0pm, Taco Hoekwater <t...@elvenkind.com> wrote:
>
>
>
>
>
>
>
>
>
> > On 11/13/2010 07:54 PM, Philipp Stephani wrote:
>
> > > stefano<stefano.fran...@gmail.com> =A0writes:
>
> > >> 2. I cannot get the current version of =A0biber to work (0.5.7). The
> > >> binary file distributed by the biber project fails to process my ref
> > >> file complaining about all sorts of missing perl modules. I tried =
=A0to
> > >> install from source, but ran into similar dependency-related problem=
s
> > >> (I posted a query on the biber forums about it).
>
> > With biber you will have a much better chance of your problems
> > actually being looked at than with bibtexu, which appears to be
> > abandonware.
>
> > Good luck,
> > Taco
>
> Update:
>
> After the failed installation of biber from source (but almost
> completely successful installation of perl modules), I retried
> installing the self-contained biber binary. Now it runs to completion
> when processing the sample file I posted earlier. Unfortunately, it
> makes things worse. Now all accented characters are corrupted. See
> below for the .bbl file.
> More and more puzzled.
>

For future reference: Thanks to Philip Kime of biber the problem is
now solved. Using the development version of biber (0.60 beta) with
the flag --bibencoding=3DUTF8 I am now able to parse the bib file
correctly.

Thanks to everyone for their help.

Stefano