It's my job to know about C++, but I'm not a C++ programmer.  That is,
producing C++ source code for nontrivial projects is not how I spend my
time.  Yet I routinely offer advice to C++ programmers about how to write
good code, and there is some evidence that some of them listen to me.  From
time to time, I need a reality check.  This is one such time.

I got the following email query from a reader.  It boils down to asking
whether it's really reasonable to expect C++ programmers to use STL
algorithms instead of writing their own loops, a topic I address in my
"Effective STL," where I argue that it often is.  But the reader puts the
question so much better.  It's long, but very much worth reading, IMO:

  One of the things that I have noticed in your Effective STL book,
  Stroustup's books, several of Herb Sutter's books, Bruce Eckel's books,
  and Matt Austern's "Generic Programming") is that most (if not all) of
  the C++ writer/developers I respect are encouraging their readership to
  embrace Generic Programming, particularly with regard to using algorithms
  -- especially in the area of replacing manual looping with algorithm
  calls.  Now, you don't have to convince me that this is a good idea.
  I've read the discussions, and so far I am in agreement with the majority
  -- at least, I *want* to be.  I am currently working on a project where I
  use a lot of standard library containers.  So in the interest of "doing
  things better", I tried to use algorithms instead of manual loops.  I had
  a really difficult time making this work.  In fact, the features I tried
  to implement with algorithms were different enough from *any* of the
  examples I found, that I finally just gave up and went back to manual
  looping.  Now, I did not give up easily (as I generally don't).  I put
  days (weeks?) into trying to figure this out.  I don't remember the exact
  problem I was trying to solve (it was a while back), but I do recall that
  the manual loop wasn't all that complicated.  I also recall being
  frustrated that I spent so much time on it and *still* was unable to make
  an algorithm do the job.  The feeling I came away with from this was
  that, sure, using algorithms *sounds* like the right thing to do, but in
  practice I can write in 30 minutes with a manual loop something I
  couldn't figure out in *days* using an algoritnm.  Function Objects
  seemed to overcomplicate the code, for one thing.  For another thing, I
  just couldn't get the algorithm to do the job I wanted done.

  Herb Sutter recommends using lambda expressions from the Boost library to
  automatically generate function objects and make algorithm calls cleaner.
  I have looked a little at that (and will look in more detail later), and
  it shows promise.  However, I am a little skeptical because of my past
  experience.  I'm no genius, but I consider myself to be a reasonably
  above average programmer.  I have a Bachelor's in CS from 10 years ago,
  have been working in the industry ever since.  I'm no newbie.  I have a
  good grasp on OOP, but am also aware that it is not a panacea.  I have
  written a fair bit at a low level in assembly and C, but have also
  written a lot of Perl, Java, and of course C++.  I've done a fair bit of
  studying on software design (including Design Patterns, etc).  One of the
  things I *like* about C++ is that it is a multi-paradigm language --
  since I'm a pretty pragmatic person, and I believe in knowing many things
  well, and using the right tool/method for the right job.  Thus you could
  understand why I want to master the whole generic programming thing.

  Can you point me to some practical software that uses generic programming
  in C++ extensively (perhaps a good open-source project)?  Even better
  would be a source for a ton of examples of the nature: "Here is how a
  programmer would probably write this using a loop.  And here is the same
  solution using an algorithm."  I realize that you have a number of
  similar examples in your Effective STL book.  I guess I'm looking for
  something more extensive.  I obviously seem to be missing something,
  since so many of the "top dogs" are leaning this direction.  But for some
  reason I'm not getting it.  Whatever the source, it would be *really*
  nice to find an entire resource devoted to "you *used* to do it this way,
  using algorithms you do it *this* way".  So far I have found no such
  resource.  Item 43 of "Effective STL" (as well as some of the other
  items) were helpful, but I guess I just need something more.  Do you have
  any suggestions?

>From my perspective, there are two questions here.  One is, as I said,
whether it is practical to replace many manual loops with algorithm calls.
The other is where people can find open-source examples of real systems
that actually do it.

The first question, especially, should really be answered only by people
who spend a lot of time working on real C++ systems.  I welcome comments on
both questions, as I hope to be able to enlighten not only my reader, but
also myself.

Thanks,

Scott

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

>>From my perspective, there are two questions here.  One is, as I said,
> whether it is practical to replace many manual loops with algorithm calls.

IME, no.

IME, and I know most people will disagree with me, the most effective 
solution is to avoid node base container and iterators and use indicies 
everywhere.

Makes coding trivial, short and unlikely to be prone to hard to catch 
errors like silently invalidated iterators or iterators going out of 
range (ok, I know that some STL implementation provide runtime checks, 
but thing is that these kinds of error are much less likely to happen 
with indicies).

I believe that all that "for_each" madness was cause by two factors:

- it looks "cool" and "advanced"

- it is annoying to create for loop for STL containers as iterators 
syntax is usually way too ugly

for(std::vector<std::string>::iterator it = c.begin(); it != c.end(); it++)

I guess this is something that nobody really likes... (I think that at 
the moment "auto" extension is introduced, all that "for_each" stuff 
will be silently forgotten in year or two).

As for me, I prefer easy to maintain variant

for(int i = 0; i < c.GetCount(); i++)


Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Yes, the issues you raise are quite fundamental.
Talking from the practical perspective:

I find myself using STL algorithms in those situations where they are 
*really* obvious. Copying, sorting or filling arrays, finding min and 
max of a given sequence, etc. - these are things that you can do by just 
picking the tool from the box and using it where it immediately fits.
(or, getting the big nail and just smashing it in with one stroke - this 
might be a good comparison as well)

Notice important words above: "immediately", "one stroke". This is the 
border behind which the whole idea breaks - and also the problem of your 
reader, I think. If the tool does not fit immediately, I just don't 
bother tinkering with it. This applies most often to std::for_each, for 
example - any time I thought of using it, it would require dragging some 
binders or composers as well. Yes, it is cool to write funky code (but 
without *true* lamda in the language it is still just pretending ;) ), 
but... in practice we don't write code only for ourselves. Working in a 
team means writing for others as well. I would not like to *have* to 
deal with somebody else's funkiness, so I don't impose mine on others - 
I believe it works very well in a long run.

If the tool does not fit immediately out of the box, I don't bother.
I see no reason for your reader to get inferiority complexes because of 
that as well.

Of course, this should not prevent anybody from reading and learning STL 
in its entirety - knowing the tools and deciding when to use them (or 
not) is always much better than not using the tools just because of not 
knowing them.


-- 
Maciej Sobczak : http://www.msobczak.com/
Programming    : http://www.msobczak.com/prog/

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Hi Scott,

/* snip */

> >From my perspective, there are two questions here.  One is, as I said,
> whether it is practical to replace many manual loops with algorithm calls.
> The other is where people can find open-source examples of real systems
> that actually do it.

> The first question, especially, should really be answered only by people
> who spend a lot of time working on real C++ systems.  I welcome comments on
> both questions, as I hope to be able to enlighten not only my reader, but
> also myself.

As for me, I program in C++ for more than five years by now, for
industry and for academic purposes, and I still prefer manual
implementation; mainly, because most algorithms found in the STL are
so trivial that I do not win anything when using them; the problem can
be solved by a similar trivial manual loop, and I prefer the explicit
form over the "hidden" algorithmic approach; the manual implementation
typically fits into three to five lines, and is IMHO more readable and
down to earth, and even easier to debug. A corresponding implementation
using algorithms is often of the same size, so I don't win anything.

The only exception where I have made use of the STL is the area of
sorting algorithms; a merge or a quick sort implementation requires
more than a trivial amount of lines, and you do make mistakes there,
so the STL version is really more convenient. If the problem at hand
would have been for industrial applications, I might have written
my own code here as well should I have found some weaknesses of the
STL version (eg. critical timing, etc.).


So long,
        Thomas


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <3kovmgFvehqvU1@individual.net>, Mirek Fidler <cxl@volny.cz> 
writes
>I believe that all that "for_each" madness was cause by two factors:
>

Was that really what the quoted person was talking about? I got the 
impression that what he meant was adding algorithms rather than using 
for_each.

-- 
Francis Glassborow      ACCU
Author of 'You Can Do It!' see http://www.spellen.org/youcandoit
For project ideas and contributions: http://www.spellen.org/youcandoit/projects


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Scott Meyers wrote:
>..
>whether it is practical to replace many manual loops with
>algorithm calls.

Let's say, If I already know how to program in C++, why
bother spending longer time to learn STL, more complex than the
language itself and contains many exceptional rules, no less
pitfalls but less flexibility, hard to understand compiler error
report.., and vaguely documented (for my encountered problems).

So it is not practical to me for now and the near future.

IMHO, template modifies the language. It should be mostly hidden.
The wording "replace many manual loops with algorithm calls."
looked to me is changing the language.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Scott Meyers wrote:

    [...]
> It boils down to asking whether it's really reasonable to
> expect C++ programmers to use STL algorithms instead of
> writing their own loops,

Which is, IMHO, the wrong question.  If a standard algorithm
fits the job, use it.  If it doesn't don't try to twist it to do
something it wasn't designed to.  And don't fall into the trap
of using for_each whenever none of the other algorithms fit,
just because it is cool.

On the other hand, instead of just writing a loop, it's often
worth considering writing an algorithm, and calling that, rather
than just coding the loop directly.  Of course, the algorithm
may have a hand written loop.  But that isn't really the point.
The point is that in the application specific code, you've given
the thing a name, and that it is usable elsewhere.  And that the
subtilities of the algorithm don't get buried down (or worse,
forgotten) in the subtilities of the application specifics.

Of course, if the standard algorithm does fit, then use it.  If
you need a linear search, use find or find_if, even if it means
writing a special predicate.  And of course, who knows.  Maybe
the predicate will be usable elsewhere, too: more than one of my
predicates has started out in an unnamed namespace, only to end
up in a header file of its own.  (The CRC and MD5 accumulators
come to mind.  For use with std::accumulate.  But also Predicate
wrappers for std::ctype functions.)

In the end, it's just plain old functional decomposition, only
more so.  Functional decomposition works.  Breaking out
algorithms, predicates, etc. works even more so.  Functional
decomposition involves a little more work up front.  The STL way
more so.  Functional decomposition saves a lot of work in the
end.  The STL way more so.

    [...]
> From my perspective, there are two questions here.  One is, as
> I said, whether it is practical to replace many manual loops
> with algorithm calls.

It's pratical to decompose your code into the application
specific part, and the algorithms.  And it seems to pay off in
the end.  About the only real problem is all of the work-arounds
you need because you're constantantly dealing with two
iterators, instead of a single object.  (This makes designing
new iterators particularly difficult.  Because when you're not
iterating over a fixed container, you frequently don't know the
end until you get there.  And it makes chaining of algorithms,
where one algorithm specifies the sequence over which the next
one iterates, unnecessarily complex as well.)

What you do have to realize is the set of useful algorithms
depends largely on the application domain.  I don't know what
application domain Stepanov had in mind when he conceived the
STL, but there are a lot of algorithms in the standard that are
completely useless for me.  But that's not really the point.
The point is that it is easy to write your own algorithms, and
that factoring the algorithm out of the application specific
software has definite advantages.

--
James Kanze                                           GABI Software
Conseils en informatique orient�e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Scott Meyers <Usenet@aristeia.com> writes (on behalf of one of his readers):

>   ....most (if not all) of the C++ writer/developers I respect are
>   encouraging their readership to embrace Generic Programming,
>   particularly with regard to using algorithms -- especially in the
>   area of replacing manual looping with algorithm calls.  Now, you
>   don't have to convince me that this is a good idea.  I've read the
>   discussions, and so far I am in agreement with the majority -- at
>   least, I *want* to be.

That certainly sounds familiar ;-)

>   I am currently working on a project where I
>   use a lot of standard library containers.  So in the interest of "doing
>   things better", I tried to use algorithms instead of manual loops.  I had
>   a really difficult time making this work.  In fact, the features I tried
>   to implement with algorithms were different enough from *any* of the
>   examples I found, that I finally just gave up and went back to manual
>   looping.  

This part is interesting.  It isn't the usual complaint that you hear
about the algorithm interface, namely, that it's cumbersome to use.
There's some validity to _that_ assertion, which is why my next
library (and the MPL, incidentally) uses an interface that accepts the
entire range to be operated on in one object instead of working on
"two separate iterators."  However, it's also the reason Boost
accepted Eric Niebler's FOREACH macro
(http://boost-sandbox.sourceforge.net/vault/index.php?directory=eric_niebler,
including docs).  [1]


>   Now, I did not give up easily (as I generally don't).  I put
>   days (weeks?) into trying to figure this out.  I don't remember the exact
>   problem I was trying to solve (it was a while back), but I do recall that
>   the manual loop wasn't all that complicated.  I also recall being
>   frustrated that I spent so much time on it and *still* was unable to make
>   an algorithm do the job.  

Often the algorithms you need are lurking there, but it can be
difficult to identify them unless you're really familiar with what's
available and how to use it.  That said, unless your reader can
remember what s/he was trying to do, it's hard to say.  The STL
doesn't claim to cover everything you might want to do with sequence
traversal, though the aglorithm suite does cover most of the
fundamentals.

>   The feeling I came away with from this was
>   that, sure, using algorithms *sounds* like the right thing to do, but in
>   practice I can write in 30 minutes with a manual loop something I
>   couldn't figure out in *days* using an algoritnm.  Function Objects
>   seemed to overcomplicate the code, for one thing.  

Unless you have a good library at your disposal to generate them with,
they can.

>   For another thing, I just couldn't get the algorithm to do the job
>   I wanted done.

.....which seems to be the crux of your reader's problem.  So we need to
know what the job was.

>   Herb Sutter recommends using lambda expressions from the Boost library to
>   automatically generate function objects and make algorithm calls cleaner.
>   I have looked a little at that (and will look in more detail later), and
>   it shows promise.  However, I am a little skeptical because of my past
>   experience.  

What past experience?  Your reader goes on to talk about his or her
qualifications and openness to ideas, but doesn't tell us what
happened.

Anyway, for what it's worth: I sometimes find Boost.Lambda
frustrating.  The library is going on 5+ years old and is showing its
age.  Heck, the Boost/TR1 Bind library, which is less powerful
overall, is much easier to use in some ways that haven't yet made it
into Boost.Lambda (your reader should try using Bind).  We've learned
a great deal about how to do these things in C++ since it was written,
and I and quite a few others have been waiting eagerly for the
so-called "Lambda/Phoenix Merger"
(http://article.gmane.org/gmane.comp.lib.boost.devel/128254) which
should be a vast improvement.

<snip>

>   One of the things I *like* about C++ is that it is a
>   multi-paradigm language -- since I'm a pretty pragmatic person,
>   and I believe in knowing many things well, and using the right
>   tool/method for the right job.  Thus you could understand why I
>   want to master the whole generic programming thing.
>
>   Can you point me to some practical software that uses generic programming
>   in C++ extensively (perhaps a good open-source project)?  

Well, the reader shouldn't equate generic programming with STL
algorithms.  That's a good place to start, but GP is a much broader
idea.  Just take a look at the Boost.Graph library, which is IMO one
of the most mature expressions -- and uses -- of generic programming
anywhere: a real tour de force.  However, I guess s/he's probably
asking you to point to _applications_.  Well, if you look at the
questions on the Boost.User mailing list you can see that *lots* of
people are using the graph library every day:
http://www.google.com/search?q=graph+site%3Alists.boost.org%2Fboost-users%2F

Also, you can look through the prototype "who's using Boost" pages and
find several mentions of the lambda library, the graph library, many
uses of Boost.Bind and Boost.Function, and countless others, all of
which are generic components and many of which are likely to be used
with STL algorithms:
http://www.boost.org/regression-logs/cs-win32_metacomm/doc/html/who_s_using_boost_.html


>   Even better would be a source for a ton of examples of the nature:
>   "Here is how a programmer would probably write this using a loop.
>   And here is the same solution using an algorithm."  I realize that
>   you have a number of similar examples in your Effective STL book.
>   I guess I'm looking for something more extensive.  I obviously
>   seem to be missing something, since so many of the "top dogs" are
>   leaning this direction.  But for some reason I'm not getting it.

Given the current algorithm interface and many of the available tools,
the use of an algorithm that is doing a simple linear traversal is
often not going to be easier to read than a for loop (or, maybe
better, a FOREACH loop).  That's something, as I've been hinting at,
that may improve over time.  However, there is a large category of
sequence algorithms that, even with today's interface and tools, I
would never want to write by hand.  I've starred some fo those from
http://www.sgi.com/tech/stl/table_of_contents.html (yes, a couple of
those listed below aren't in the standard -- I've tried to avoid
starring those):


   1. Non-mutating algorithms
         1. for_each
         2. find
         3. find_if
*        4. adjacent_find
*        5. find_first_of
         6. count
         7. count_if
*        8. mismatch
         9. equal
*       10. search
        11. search_n
*       12. find_end 
   2. Mutating algorithms
         1. copy
         2. copy_n
         3. copy_backward
         4. Swap
               1. swap
               2. iter_swap
               3. swap_ranges 
         5. transform
         6. Replace
               1. replace
               2. replace_if
               3. replace_copy
               4. replace_copy_if 
         7. fill
         8. fill_n
         9. generate
        10. generate_n
        11. Remove
               1. remove
               2. remove_if
               3. remove_copy
               4. remove_copy_if 
*       12. unique
*       13. unique_copy
        14. reverse
        15. reverse_copy
*       16. rotate
*       17. rotate_copy
*       18. random_shuffle
*       19. random_sample
*       20. random_sample_n
*       21. partition
*       22. stable_partition 
   3. Sorting
         1. Sort
*              1. sort
*              2. stable_sort
*              3. partial_sort
*              4. partial_sort_copy
*              5. is_sorted 
*        2. nth_element
*        3. Binary search
*              1. lower_bound
*              2. upper_bound
*              3. equal_range
*              4. binary_search 
*        4. merge
*        5. inplace_merge
         6. Set operations on sorted ranges
*              1. includes
*              2. set_union
*              3. set_intersection
*              4. set_difference
*              5. set_symmetric_difference 
         7. Heap operations
*              1. push_heap
*              2. pop_heap
*              3. make_heap
*              4. sort_heap
*              5. is_heap 
         8. Minimum and maximum
               1. min
               2. max
               3. min_element
               4. max_element 
         9. lexicographical_compare
        10. lexicographical_compare_3way
*       11. next_permutation
*       12. prev_permutation 
   4. Generalized numeric algorithms
         1. iota
         2. accumulate
*        3. inner_product
*        4. partial_sum
*        5. adjacent_difference
         6. power 

So there's a cluster in the beginning that might be trivial enough to
write out by hand, but when I really think about it, I wouldn't want
to write most of those myself either ;-)

And that's just the expressiveness issue.  When you consider the
efficiency issue, the argument for even simple algorithms gets
stronger.  Libraries can do loop unrolling or more sophisticated
things [Austern98] to make those simple algorithms better than your
hand-coded loops -- if you care about efficiency, that is.  

Another factor is that the STL algorithms and iterators are aimed at
the highest possible speed.  Many loops in an application don't need
to be that fast.  In fact, sometimes it's more important to reduce
code size, which is where components like Boost.Function and
any_iterator (here's a version from Adobe:
http://opensource.adobe.com/classadobe_1_1any__iterator.html) that use
type erasure come in handy if you want to stay within the paradigm.

>   Whatever the source, it would be *really* nice to find an entire
>   resource devoted to "you *used* to do it this way, using
>   algorithms you do it *this* way".  So far I have found no such
>   resource.  Item 43 of "Effective STL" (as well as some of the
>   other items) were helpful, but I guess I just need something more.
>   Do you have any suggestions?
>
>>From my perspective, there are two questions here.  One is, as I said,
> whether it is practical to replace many manual loops with algorithm
> calls.

Sometimes.  Usually the algorithms that are worth the trouble are much
more interesting than a loop.  Better libraries and interfaces will
improve that situation a little.

> The other is where people can find open-source examples of real systems
> that actually do it.

I hope I gave you some useful pointers above.

> The first question, especially, should really be answered only by people
> who spend a lot of time working on real C++ systems.  

Wow, I'm not sure whether you think library authors qualify.  I wish
you had said that at the beginning! ;-)

> I welcome comments on both questions, as I hope to be able to
> enlighten not only my reader, but also myself.

Hope this helped.

Footnotes: 

[1] This library was accepted long ago
    (http://www.boost-consulting.com/boost/more/formal_review_schedule.html);
    I'm not sure why it hasn't been added yet.

   [Austern98] Matthew H. Austern, *Segmented Iterators and
   Hierarchical Algorithms*, 1998. Lecture Notes In Computer
   Science; Vol. 1766 Selected Papers from the International
   Seminar on Generic Programming, Pages: 80 - 90,
   ISBN:3-540-41090-2 http://lafstern.org/matt/segmented.pdf
-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Scott Meyers wrote:
> 
> The first question, especially, should really be answered only by people
> who spend a lot of time working on real C++ systems.  I welcome comments on
> both questions, as I hope to be able to enlighten not only my reader, but
> also myself.
> 

I think the learning curve for function objects is quite high.  I've 
only browsed boost::lambda docs as Borland's bcc32 5.6.4 can't compile 
it which we currently use, and I think it does help with creating 
in-place function objects for use in algorithms, but is subtly different 
enough from the manual loop syntax that it isn't such a simple learning 
curve.  I'd like to be able to write something like:

for_each(container.begin(), container.end(), _1->function());

But I don't believe that is possible without language support.  I 
believe the lambda expression would be

for_each(container.begin(), container.end(), _1 ->* &myclass::function);

(But not sure as can't check with compilation).

We are currently stuck with boost::bind which gives us

for_each(container.begin(), container.end(), bind(&myclass::function, _1));

I took up to a year for me to be confident enough with bind to decide it 
may be better than a hand-written loop.  But there are cases where it 
just looks complicated.  e.g. some of our data structures have a 
'get_name' member function with returns a string.  We want to find which 
one has a particular name and the bind expression comes out to be:

find_if(m_ExistingExcipients.begin(), m_ExistingExcipients.end(),
         bind(equal_to<string>(), bind(&Excipient_c::GetDescription, 
_1), Name));

The lambda equivelant of this may be simpler, but this certainly isn't 
obvious to people who are un-familiar with this expression and its hard 
to convice them that it is better to use this than a hand-written loop, 
especially with the current errors you get from the compiler if you make 
a mistake in your expression.

Sorting them by description gives us

std::sort(Excipients.begin(), Excipients.end(),
           bind(less<string>(), bind(&Excipient_c::GetDescription, _1),
                                bind(&Excipient_c::GetDescription, _2)));

Eric Niebler's BOOST_FOREACH macro has been accepted and will hopefully 
appear in boost-1.34 which may well simplify things as with that, the 
above could be

shared_ptr<Excipient_c> Excipient;
BOOST_FOREACH(Excipient, Excipients)
{
   if (Excipient->GetDescription() == Name)
   {
     break;
   }
}

Which is much more readable IMHO even if it is using a macro.  The 
disadvantage of BOOST_FOREACH is that I don't think you can get access 
to the iterator if Excipient which the std::algorithms and hand-coded 
loops give you.

I try to use algorithms now where possible (but only using bind, not 
lambda yet until Borland finally upgrade their compiler) and lambda does 
simplify things but I still believe that lambda and bind have too steep 
a learning curve to be immediately attractive.  As mentioned, the lambda 
library is very powerful, but the syntax is similar yet different to 
what you'd write in a hand-written loop that it isn't immediately 
obvious what you need to change from the hand-written version to turn it 
in to a lambda expression.

Another library not mentioned here is boost::range which can simplify 
the expressions slightly for both hand-written loops and algorithms as 
you don't need both begin and end.

Cheers

Russell

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On 27 Jul 2005 Scott Meyers wrote:
>I got the following email query from a reader.  It boils down to asking
>whether it's really reasonable to expect C++ programmers to use STL
>algorithms instead of writing their own loops, a topic I address in my
>"Effective STL," where I argue that it often is.  But the reader puts the
>question so much better.  It's long, but very much worth reading, IMO:

I remember some months ago there was a long thread and discussion 
in this newsgroup about this topic, algorithms or loops. What I 
have observed as a lurker is: There were questions about the 
meaning and the right way to use the algorithms. How did the 
functional objects look like? Their source was not at the same 
place as the one-liner algorithm. And so on. May be it just 
doesn't look very common and custom. That is part of the problem. 

What everybody in the thread did first was to show the corresponding 
loop, a three-liner, and nobody asked about its meaning. Everybody 
understood immediately. Imagine the not so skilled collegues who 
may have to understand and change your code. That is part of the 
answer to your questiion. 

Harald


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Scott

I have been programming c++ for a living for a number of years and using the STL for quite a while.  I've read quite a 
few of the texts which discuss how to take advantage of this code (including yours : ) . I, too, often debate this issue 
and haven't decided whether some of these techniques are really worth while.

For example, I have some production code in a system I maintain which uses a map whose data type is a vector of Boost 
smart pointers.  With the STL containers this is trivial to set up and saves a lot of time.

When I insert something into the vector for an entry I don't want to have duplicates so I use the following code to see 
if I have an existing entry before adding a new one.

--------code below

Mbr_Jrnl_Data_List&	memberList	= mJrnl_Data_List[newData->mJRNL_ID];

Mbr_Jrnl_Data_List::iterator i = std::find_if(memberList.begin(),
	memberList.end(), boost::bind(&JRNL_Data::Matches, _1, boost::ref(newData)));

if (i == memberList.end())
	memberList.push_back(newData);

--------code above

To me, this seems easier an more straight forward than writing the loop myself.

However, I have some other code where I want to extract certain records from the vector which belongs to a certain
key in my map and add those entries to a new vector.  After much effort, I have the following code.

----------code below

std::remove_copy_if(i->second.begin(), i->second.end(), std::back_inserter(layoutTrans),
	boost::bind
		(std::not_equal_to<JRNL_Data::RequestTypeCodes>(), boost::bind(&JRNL_Data::GetRequestType, _1), JRNL_Data::e_layout_req));

--------code above

While this works and is actually fairly compact for what I need to do and did not require me to write any code other 
than the to calls STL algorithms it took a lot longer to figure this out than it would have taken to write the loop 
myself.  I am going with this version for now because it requires less of 'my' code and also because I hope that if I 
work with more of this type code I will get to be more comfortable with it.  However, when someone else comes to work on 
this code in the future he/she may not find this 'intuitively obvious'.  I do have an extensive comment describing what 
I am doing in this method.

Dave Riedel


Scott Meyers wrote:
> It's my job to know about C++, but I'm not a C++ programmer.  That is,
> producing C++ source code for nontrivial projects is not how I spend my
> time.  Yet I routinely offer advice to C++ programmers about how to write
> good code, and there is some evidence that some of them listen to me.  From
> time to time, I need a reality check.  This is one such time.
> 
> I got the following email query from a reader.  It boils down to asking
> whether it's really reasonable to expect C++ programmers to use STL
> algorithms instead of writing their own loops, a topic I address in my
> "Effective STL," where I argue that it often is.  But the reader puts the
> question so much better.  It's long, but very much worth reading, IMO:
> 
snip

> 
> Thanks,
> 
> Scott
> 
>       [ See http://www.gotw.ca/resources/clcm.htm for info about ]
>       [ comp.lang.c++.moderated.    First time posters: Do this! ]
> 

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Scott Meyers wrote:
> I got the following email query from a reader.  It boils down to asking
> whether it's really reasonable to expect C++ programmers to use STL
> algorithms instead of writing their own loops, a topic I address in my
> "Effective STL," where I argue that it often is.  But the reader puts the
> question so much better.  It's long, but very much worth reading, IMO:

The algorithms and the for loops are both tools.  Use the tool that
makes sense for the task, based on your requirements, proficiency,
environment, ...

I like to use my power screwdriver, when it is handy, and it fits.  It
lets me get a lot of work done quickly, and it has features (such as a
torque setting) that are useful.  However, I sometimes end up using the
screwdriver on my pocketknife instead.

I use copy, find, and find_if frequently.  I'm much more likely to use
a pre-written sort (or even lower_bound) than to write my own.

If the body of a for loop is a few lines long, I probably won't break
it out and use for_each, until I need to reuse it, and then, in most
cases, I end up writing a function that contains the actual loop, so
that the client code calls
   ProcessRange(begin, end);
rather than
   for_each(begin, end, ProcessElement);

On the other hand, if ProcessElement is useful in other contexts, the
body of ProcessRange might end up being a single for_each call.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <MPG.1d50a59dd15264319897e0@news.hevanet.com>,
 Scott Meyers <Usenet@aristeia.com> wrote:

> From my perspective, there are two questions here.  One is, as I said,
> whether it is practical to replace many manual loops with algorithm calls.
> The other is where people can find open-source examples of real systems
> that actually do it.

I don't find for_each useful.  But I do find most other algorithms in 
<algorithm> useful, even simple ones like copy.

for_each doesn't save the coder any work because it is just as hard or 
harder to put the body of the loop into a functor as it is to just write 
the loop.  If you already have such a functor laying around for other 
purposes, for_each can be great.  But creating a functor just so you can 
use for_each will often waste more time than it saves.

That being said, I'd love to see these cousins of for_each:

// Call f(first, first+k) for each permutation of
//  [first, last) taken k items at a time.
template<class BidirectionalIterator, class Function, class Size>
Function
for_each_permutation(BidirectionalIterator first,
                     BidirectionalIterator last,
                     Size k, Function f);

// Call f(first, first+k) for each reversible permutation of
//  [first, last) taken k items at a time.
//  "reversible" may be the wrong word here.  The
//  intent is that the reverse of a permutation doesn't
//  count as a permutation, and is thus not applied.
template<class BidirectionalIterator, class Function, class Size>
Function
for_each_reversible_permutation(BidirectionalIterator first,
                                BidirectionalIterator last,
                                Size k, Function f);

// Call f(first, first+k) for each circular permutation of
//  [first, last) taken k items at a time.
template<class BidirectionalIterator, class Function, class Size>
Function
for_each_circular_permutation(BidirectionalIterator first,
                              BidirectionalIterator last,
                              Size k, Function f);

// Call f(first, first+k) for each reversible circular permutation of
//  [first, last) taken k items at a time.
template<class BidirectionalIterator, class Function, class Size>
Function
for_each_reversible_circular_permutation(BidirectionalIterator first,
                                         BidirectionalIterator last,
                                         Size k, Function f);

// Call f(first, first+k) for each combination of
//  [first, last) taken k items at a time.
template<class BidirectionalIterator, class Function, class Size>
Function
for_each_combination(BidirectionalIterator first,
                     BidirectionalIterator last,
                     Size k, Function f);

The main difference between these, and std::for_each, is that these do 
useful, non-trivial work for you, making the effort to create the 
functor worth it.

-Howard

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Scott Meyers wrote:
>>From my perspective, there are two questions here.  One is, as I said,
> whether it is practical to replace many manual loops with algorithm calls.
> The other is where people can find open-source examples of real systems
> that actually do it.
>  
I've worked on some reasonable-sized C++ projects. When I want something
which an STL function is actually writen for (sort, min, max, etc), then
I'll use it. The main problem with most other tiny loops is even if they
would map nicely to (say) for_each, they require writing a functor,
which has to be declared out-of-line, so unless it's going to be used
alot, its easier to just write the loop out (and will be even easier
once we get "auto").

I mean, considering the body of for_each is basically:
      for ( ; __first != __last; ++__first)
        __f(*__first);

If writing f is going to take any longer than one line, then it's really
not worth using it from a code-length point of view :)

In my own code I've used boost::lambda to great effect, but I find
myself unwilling to try to introduce it into other people's large
projects as it consists of a large amount of very complex C++ code I
don't understand very well (not that I'm insulting it of course, but I
like to be sure I understand code I'm putting into projects..)

Chris

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

The project(s) I generally work on are embedded test and measurement
equipment. IMO, STL containers, algorithms, generic programming in
general, Boost, etc. has saved me tons of time. I use std::for_each so
much it looks kinda funny in some places in the code. I believe that a
mastery of these techniques will make you much more productive in the
long run. I work with some people who cling to the 'old ways' like
their lives depend on it. I find that most of the other programmers in
my group make more limited use of these techiniques because their
understanding of them is more limited. One or two guys want to use the
STL more often but can't get their brains around it sometimes. A couple
of the guys don't want anything to do with it and hence will never
understand how to use the STL, by choice. Myself, I'm more of an STL
and generic programming zealot. The project(s) I work on are therefore
overflowing with Boost, Loki, STL, etc. The other guys in my group make
a more limited contribution to the code because they're much slower at
producing it than myself. I believe that their lack of STL
understanding and practice is one of the reasons.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Hi Scott!

I have 8 years C++ experience from telecom realtime systems on open
platforms
(UNIX). I have the following to say:

STL is not an abstraction, it's an abomination. STL is a liability in
any software
development project.
- Being completely non-intuitive, it constantly leads even seasoned and
clever
  programmers into serious pitfalls and bad design decisions.
- Designers have to rely on 3PP implementations trying to be to clever
when using
  memory etc. I've used SUN's and it sucks when things like
  deque< deque< vector < string > > > is to be used.
- An abstraction that creates more questions than it's supposed to
answer is
  nothing but a joke.

If you don't take my word for it I think the following is proof enough:
- I search on C++ and STL in Amazon and find over 50 books with STL
tips and trix
  as their only angle!!!
- I search for STL in ONE periodical ( C/C++ Users Journal ) and I find
over
  400 articles!!!

When did STL suckage become taboo?
Can't anyone tell Bjarne to go back to the drawing board on this one?

BRs
/Sune


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Francis Glassborow wrote:
> In article <3kovmgFvehqvU1@individual.net>, Mirek Fidler <cxl@volny.cz> 
> writes
> 
>>I believe that all that "for_each" madness was cause by two factors:
>>
> 
> 
> Was that really what the quoted person was talking about? I got the 
> impression that what he meant was adding algorithms rather than using 
> for_each.
> 

Well, reading the other responses perhaps I got it wrong.

Anyway, term "manual loop" feels like it is about trivial loops 
implemented via STL using functors, adaptors, lambdas and hell knows what.

It certainly is not about complex and hard to implement algorithms like 
std::sort.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Scott Meyers wrote:
[loads of reasonable background snipped]

>>From my perspective, there are two questions here.  One is, as I said,
> whether it is practical to replace many manual loops with algorithm calls.
> The other is where people can find open-source examples of real systems
> that actually do it.

I don't know of open-source examples of real systems using algorithms
a lot and, honestly, I doubt that you will be able to find many of
them. However, still being a strong advocate of the use of algorithms
I will address the first question.

The short version of my answer is this: Generic Programming is the way
to go for all algorithmic needs but we don't yet have the necessary
tools in place for their general practical use.

There are many problems with the current set of algorithms in the
Standard C++ Library which hamper their use in everyday work. Here
are the problems and the cure I envision:

- STL addresses only a very specific albeit basic set of algorithms.
   If you need something different than an algorithm on a sequence you
   will not get any help from the standard C++ library. This is a well
   recognized problem which is addressed at least partially by people
   identifying the basic concepts for other problem domains and creating
   corresponding algorithms. The Boost Graph Library is an example of
   addressing a different problem domain than sequences. Since the
   algorithmic structures differ widely, it is only occasionally viable
   to build upon algorithms from another problem domain. For the rest of
   this discussion I will assume that we are only concerned with sequence
   algorithms.

- The algorithms library is quite incomplete although most of the basic
   algorithms are all there. It is just a lack of some need combinations
   or a lack of flexibility. For example, to use 'mismatch()' you need
   to know which of the two sequences is shorter and pass it in as first
   sequence. There is no overload taking two pairs of iterators or two
   ranges which makes it effectively impossible to use 'mismatch()' on
   two input sequences which are quite frequent in my code because it is
   often trivial to create an input iterator but not that easy to create
   even a forward iterator. Another example is the lack of something
   like 'copy_until()'. Of course, a 'find_if()' followed by a 'copy()'
   on the subsequence does the trick - except that it again does not
   cover the input iterator case.

- For many typical cases the algorithms are relatively hard to use due
   to their need for pairs of iterators. Although the flexibility to
   operate on subsequences is essential for a fundamental layer, it
   makes their quite hard and unnecessarily wordy. It should be
   possible to pass a "range" which exposes a 'begin()' and an 'end()'
   (i.e. containers happen to be range objects) in addition to overloads
   taking sequences. Although these algorithms would mere extract the
   sequence and forward it to a corresponding algorithm, this has a huge
   impact on usability and also allows algorithm chaining, i.e. passing
   the result of one algorithm immediately as the input of another
   algorithm.

- The STL algorithms erroneously operate on a concept combining access
   and traversal. There are loads of problems resulting from this and
   I have written about many of them in the past (search for "property
   map" and/or "data accessor" in comp.lang.c++*). The cure is for the
   fundamental algorithms to operate on cursors (iterator like entities
   which are used for traversal and which expose a "key" for the current
   position) and property maps (function like entities which take a "key"
   and access an object associated with this key; note, that this
   association is typically just a pointer dereference or an index
   access in a array). This separation might seem contradictory to the
   usability issue mentioned previously but it actually isn't: the
   algorithms can easily have simple wrappers operating on iterators and
   conjuring up a default property map before delegating to the more
   general algorithm. However, the needed flexibility is still there and
   can be used where needed which is more frequently the case than
   possibly expected.

- Generic algorithms are usually configured using some form of functor.
   However, given the current standard library it is pretty hard to just
   create simple functors on the fly without creating a separate function
   which kind of defeats the use of algorithms in the first place. The TR1
   provides improved functors which cure some of biggest problems.
   Something like Boost::Lambda goes a huge step further and allows easy
   creation of many functors. The real cure cannot, however, be done in a
   pure library approach and what is effectively needed are true "Lambda
   Expressions" whatever this really is in detail.

These are the most problematic areas I have found with STL in the past
including outlines of how to address these. I'm pretty sure that there
are still problems lurking once a cure for each of them is in place
but I would expect that addressing the above problems will take use much
further. The current STL was installed without any real experience and
actual use typically reveals real problems. I'm still convinced that
Generic Programming is the correct approach, although the current
interface to algorithms leaves much to be desired.

In my own code, I'm currently using a relatively pragmatic approach to
the use of algorithms: if there is something which fits easily or with
only a moderate amount of tweaking, I use it in favor of a hand-crafted
loop. Otherwise I just write the loop. After all, the intention is to
make live easier. However, I always try to figure out whether the
reason for writing the loop would be removed by the approaches outlined
above in an attempt to reveal additional problems.
-- 
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Scott Meyers wrote:
> I got the following email query from a reader.  It boils down to asking
> whether it's really reasonable to expect C++ programmers to use STL
> algorithms instead of writing their own loops, a topic I address in my
> "Effective STL," where I argue that it often is.  But the reader puts the
> question so much better.  It's long, but very much worth reading, IMO:

This is a religious issue.

My particular take on it is:
- if a suitable algorithm exists, use it.
- If the same thing needs to be done in several places to different
container objects, then it may be worth writing an algorithm for it.
- But in general, a straightforward loop is easier to read.

Hmm.  I was about to quote the thread "STL functor problem" as an
example, but actually that is quite difficult to handle with a simple
loop (because the simple loop version involves altering the container
while looping).

Nonetheless, I don't find boost::lambda easy to read (the notation for
using member functions is painful), and bind1st / bind2nd / compose,
because very difficult, very quickly.

Most of my code uses loops.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Scott Meyers <Usenet@aristeia.com> writes:
>
> >From my perspective, there are two questions here.  One is, as I said,
> whether it is practical to replace many manual loops with algorithm calls.

I guess it depends.

Lately I have to refactor more spaghetti then I'd like to, and
replacing the loops with algorithms gives me a pretty good place to
start cutting a few hudred lines long method into pieces. This kind of
algorithmisation also seems to have benefical effect on the data
structures of the module.  Because of this very effect of refactoring
data modells, I also find it good to reason about the design in terms
of algorithms.

But I don't think I'd replace a loop in a properly designed and
factored code just because using algorithms is cool.  (I have more
than enough problem with the badly designed and factored ones ;-) )

ImRe

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Scott Meyers wrote:
<snip>
> From my perspective, there are two questions here.  One is, as I said,
> whether it is practical to replace many manual loops with algorithm calls.
> The other is where people can find open-source examples of real systems
> that actually do it.
>
> The first question, especially, should really be answered only by people
> who spend a lot of time working on real C++ systems.  I welcome comments on
> both questions, as I hope to be able to enlighten not only my reader, but
> also myself.

I've had a look at the system I work on and the number of for_each is
pretty minimal. It has generally been used where the operation has
pretty limited side effects - either modifying the object it is passed,
or modifying some global data (such as writing to cout), and where any
pre/post processing doesn't have any bearing on what happens in the
loop (or what has happened in the loop).

As soon as you try to do anything more complex than that, things get
pretty hairy indeed. For an algorithm that works:

begin
  do a lot of initialisation with variables
  for each member of the set
  {
      modify a few of the variables
      maybe modify the member
  }
  fiddle about a bit more with the variables
  return some result
end

you have to create a class that:
1) Has all the variables you need defined as members
2) Initialises them all in the constructor
3) provide an operator()(member type&) which does the loop code
4) provide a final operation to do the post processing and return the
result value

This isn't impossible, but it is surprisingly painful. It's probably
something to do with the operator()(thing &) syntax, which feels
unnatural.

And none of it works if you have a loop that does
for (thing = ctr.begin(), i = 0; thing != ctr.end(); ++thing, ++i);


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Scott Meyers wrote:
> [...] The other is where people can find open-source examples of real
> systems that actually do it.

For that question: As a nonrepresentive estimate, I've put together a
script that scans all the c++ source files (ie: C|cpp|cc|H|cpp|hh) from
the slackware 10.1 source cd's for the pattern: (find_if|remove_if
for_each|bind1st|bind2nd). 

Here are the packages that contained any of these: 
blackbox-0.65.0 gcc-3.3.4 oprofile-0.8.1 kdeedu-3.3.2 kdegraphics-3.3.2
kdemultimedia-3.3.2 kdenetwork-3.3.2 kdepim-3.3.2 kdeutils-3.3.2
fluxbox-0.1.14 wv2-0.2.2 lftp-3.0.13 abiword-2.0.12 fluxbox-0.9.12
ImageMagick-6.1.9-0

Number of lines with: 
find_if 37
remove_if 10
for_each 104
bind1st 5
bind2nd 22
and this includes libstdc++ and both versions of fluxbox..

The total number of c++ source lines scanned was: 8811927

i.e. in ~8.8 Milliones lines of open source c++ code, less than 200 uses
of comparably important constructs from <algorithm> and <functional>.
Not sure if that means something...

Marco


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Quite simply, I am learning to use standard algorithms as a replacement
for loops. I can see, using a simple text search, how slowly more and
more of manual loops are replaced with for_each, transform, etc,
however I only invest a few minutes in trying to 'reshape' a manual
loop to fit an algorithm, time is money.

In general my programming style has moved towards deep callstacks with
specialized inline functions, which are amenable to binding, being the
norm. I do not know if this is good or not, but It seems to be my
'natural' progression in the face of the STL/boost libraries. IMHO it
is FAR more readable to use a(well spaced) for_each than most
equivalent loop constructs, but I think that this is a matter or style
and experience.

That being said, I have the luxery of working with modern tools, on
fresh 'modern C++' source code, so I doubt this is very representative.

JJJ


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Russell Hind wrote:

> Sorting them by description gives us
> 
> std::sort(Excipients.begin(), Excipients.end(),
>            bind(less<string>(), bind(&Excipient_c::GetDescription, _1),
>                                 bind(&Excipient_c::GetDescription, _2)));
> 
> Eric Niebler's BOOST_FOREACH macro has been accepted and will hopefully 
> appear in boost-1.34 which may well simplify things as with that, the 
> above could be
> 
> shared_ptr<Excipient_c> Excipient;
> BOOST_FOREACH(Excipient, Excipients)
> {
>    if (Excipient->GetDescription() == Name)
>    {
>      break;
>    }
> }
> 
> Which is much more readable IMHO even if it is using a macro.  The 
> disadvantage of BOOST_FOREACH is that I don't think you can get access 
> to the iterator if Excipient which the std::algorithms and hand-coded 
> loops give you.

Just to be pedantic, the second example is the equivalent of 
std::find_if, not of std::sort.  std::sort can not be written 
efficiently  using BOOST_FOREACH, as it requires mutating the underlying 
sequence in multiple passes.

   -- MJF

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> When did STL suckage become taboo?

Hehe, perhaps we should form "STL haters" club...

Anyway, there is one good thing that STL did for me:

It helped to finaly refine templates in core language and compilers.

> Can't anyone tell Bjarne to go back to the drawing board on this one?

Well, does it really matter?

As long as C++ core language does not screw my libraries, it is OK with me.

Mirek

P.S.: I hope that it is not quite off-topic: as an alternative, I dared 
to develop, maintain and use this:

http://upp.sourceforge.net/srcdoc$NTL$en-us.html

the main difference is the lack of "copy-constructible" and "assignable" 
requirements, I believe that it immediately cuts off at least 70% of 
problems that I have encountered in STL.

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

chris jefferson wrote:
> In my own code I've used boost::lambda to great effect,
> but I find myself unwilling to try to introduce it into
> other people's large projects as it consists of a large
> amount of very complex C++ code I don't understand very
> well (not that I'm insulting it of course, but I like to
> be sure I understand code I'm putting into projects..)

Yes, I've run into the same pitfall. I used boost's shared
pointer, and regex library in one of my projects. When I
passed the code onto a colleague, he had a hell of a time
getting boost setup on his system at work (permission
problems, build problems, etc). Eventually he became so
frustrated he just (sadly) recoded the regex portions of
the code in Java (which has a regex library) and then just
linked to the remaining C++.

And as to understanding boost code, I realize I'm not so
bright but understanding some of the boost code can be a
real mind frak [ref Battlestar Galactica].


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

I'm working in the financial area, programming (socket) server software
to various stock exchange terminals. I use STL extensively, and it
saves me large amounts of time. With the .NET marketing avalanche, I
took some time do think how I would reengineer the software in C#. My
conclusion was that I would be less productive with the
"high-productivity languages" (C#, Java) than I am using C++ and STL.

I see no reason to use loops when you have a STL algorithm, everyone
can figure out what std::for_each does, even a non C++ programmer. Why
do I need to repeat the same for(xx::interator i = x.begin... over and
over again? It's error prone, you have the temptation of using
copy/past all over the code, and we all know where it ends. You can use
an iterator from an instance of container as the end test of another
instance, just to mention one common mistake.

I use maps, vectors and STL algorithms to control and filter connected
users and the assets users want to view/sign. The STL code is so
trivial, so verbose, that I use it to show people that C++ with STL is
not as hard as most people think.

Example:

for_each(clients.begin(), clients.end(), CancelAllSignatures());

I think it's much better than a loop. It's obvious we will cancel all
signatures from all clients. But it's good because the code reader
doesn't need to know the implementation detail. And every good
IDE/Editor has a way to send you to the definition of
ClearAllSignatures just pressing a key.

And I sure agree we need better error messages from the compiler when
using templates.

Regards,

Rodrigo Strauss
http://www.1bit.com.br


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <3kprvgFv33nsU1@individual.net>, Mirek Fidler <cxl@volny.cz> 
wrote:

> P.S.: I hope that it is not quite off-topic: as an alternative, I dared 
> to develop, maintain and use this:
> 
> http://upp.sourceforge.net/srcdoc$NTL$en-us.html
> 
> the main difference is the lack of "copy-constructible" and "assignable" 
> requirements, I believe that it immediately cuts off at least 70% of 
> problems that I have encountered in STL.

Changes to list, set, multiset, map, multimap are already in the C++0X 
working paper which greatly reduce the assignable requirement:

http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-defects.html#276

I hope to further reduce the need for both CopyConstructible and 
CopyAssignable both in containers, and in algorithms, with the 
introduction of the rvalue reference:

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1771.html

-Howard

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Usenet@aristeia.com (Scott Meyers) wrote (abridged):
> whether it is practical to replace many manual loops with algorithm 
> calls.

Only a small fraction of them. For a given loop, it depends on the 
complexity of the algorithm and whether I need to write a predicate.

I use even simple algorithms if I don't need to write a predicate for 
them, eg rotate() and some uses of find(). I don't use for_each() because 
that usually does need a predicate, and I won't write a predicate in order 
to use find(). Some algorithms, such as the sorting and searching ones, 
are worth writing a predicate in order to use.


> The first question, especially, should really be answered only by people
> who spend a lot of time working on real C++ systems. 

That has been my job for over 10 years.

-- Dave Harris, Nottingham, UK.

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Scott Meyers wrote:

>>From my perspective, there are two questions here.  One is, as I said,
> whether it is practical to replace many manual loops with algorithm calls.
> The other is where people can find open-source examples of real systems
> that actually do it.
> 
> The first question, especially, should really be answered only by people
> who spend a lot of time working on real C++ systems.  I welcome comments on
> both questions, as I hope to be able to enlighten not only my reader, but
> also myself.

As a huge advocate of the use of STL algorithms over hand-coded loops,
but being also pragmatic when it comes to write my code, I have to add a
few things to what has been said:

1) A lot of people seem to be forgetting that using STL algorithms is
a form of "refactoring code".  People keep insisting that a hand-coded
loop is easier to read (presumably because you're seeing exactly what
the loop is doing).

However, I must disagree with such a statement in the general case;
In a simple, isolated short fragment of code, maybe a hand-coded loop
showing exactly what is going on may be easier to read.  But when a
function that could be 30 lines long with just a few if-elses and 5
calls to STL algorithms becomes 200 lines, then it is not easier to
read.

A *named* call to an algorithm reveals exactly and *up front* what
the code is going to do -- and it's easier to skip if you're, say,
searching for a bug that you happen to know is further down the code.

A hand-coded loop takes a few seconds (or a few minutes) inspecting
it to *realize* what the code is doing.

All this, in addition to the fact that in one line of code there are
less chances to introduce a bug due to a typo or an oversight  (aaahh,
how many times have I hated myself for being lazy and coding a loop and
then hours and hours of debugging later realizing that I made a typo
and, say, put the wrong name to the loop-control variable (and by
chance I got the name of another one that had been declared, and thus
the compiler didn't flag an error)

IMHO, the value of having *one* line of code for *one action* is very
high and compensates for many of the inconveniences.


2) True that item (1) above is partly contradicted if we shift the
focus of your original question from "STL algorithms vs. hand-coded
loops" to "STL algorithms vs. a hand-coded *function*".

For instance, it is really easier to argue that the second line is
better/easier-to-read than the second one:

     pos = count_if (cont.begin(), cont.end(), bind2nd(greater<int>(0)));

     pos = num_positive_elements (cont);

(of course, in the implementation of the function num_positive_elements,
I would make it a one-liner, using the first line above  :-)).

Actually, there would be an advantage in using such wrapper, even if
one is going to code with count_if and bind2nd:  you get rid of the
hardcoded <int> in the code:

template <typename Container>
int num_positive_elements (const Container & c)
{
     typedef typename Container::value_type T;
     return count_if ( ...., bind2nd(greater<T>(0)));
}

Coming back to the two one-liner alternatives to count positive values,
the STL alternative in that case has the advantage that you don't have
to write any additional code -- you're refactoring with functions that
are already there.


3) You said it beautifully in your Effective STL book, item 47:  Avoid
producing write-only code.  In fact, a lot less than the examples you
show, I would still call "write-only"  :-)

Let's say that my rule of thumb goes more or less like:  "Using an STL
algorithm is better until proven the opposite".  That is, when making
a decision, I try to maintain a strong bias towards using STL algorithms
and only resort to a hand-coded loop when I'm absolutely convinced that
using an STL algorithm is (a) completely twisted and inconvenient, or
(b) beyond my level of knowledge/skills with the STL.

Obviously, the above depends heavily on the level of familiarity with
the STL of each programmer.  While implementing copy_if in terms of
remove_copy_if may be a trivial "warming-up exercise" for you (or for
John Potter, who, as I recall, came up with a *correct* implementation
a while ago in this newsgroup), it is rather twisted for me (and I guess
for many other programmers).  Yes, my solution has been to implement
my own version of copy_if with a hand-coded loop in it  :-)


Perhaps it would be a good idea that people post *concrete examples*
of a given situation, and their choice for that particular situation;
for instance, if I have a list of Packets to be transmitted, and each
Packet has a time tag (the time at which it was dispatched to be sent),
and I'm looking for the oldest, then I'd definitely use the STL:

class cmp_timetag
{
public:
     bool operator() (const Packet & p1, const Packet & p2) const
     {
         return p1.sent_at() < p2.sent_at();
     }
};

min_element (transmit_queue.begin(),
              transmit_queue.end(),
              cmp_timetag());


If I need to count how many strings are longer than a given string,
I'd definitely use the STL:

class longer_than
{
     const string & s;
public:
     longer_than (const string & s) : s(s) {}

     bool operator() (const string & item) const
     {
         return item.length() > s.length();
     }
};

count_if (names.begin(), names.end(), longer_than("Carlos"));

Yes, writing the functor implies an amount of work up front, but the
readability of the resulting code is, IMHO, greatly improved  (plus,
I'll bet you -- 1 out of 10 times, an average programmer *will*
forget to initialize the counter to 0 in the hand-coded loop
version of the above!  :-)).


HTH,

Carlos
--

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> do I need to repeat the same for(xx::interator i = x.begin... over and

It is problem of iterator syntax, not of 'for'.

> Example:
> 
> for_each(clients.begin(), clients.end(), CancelAllSignatures());
> 
> I think it's much better than a loop. It's obvious we will cancel all
> signatures from all clients.

Is it? What CancelAllSignatures returns? Is it a functor?

> But it's good because the code reader
> doesn't need to know the implementation detail.

Impementation detail of what? for_each? Come on.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> Changes to list, set, multiset, map, multimap are already in the C++0X 
> working paper which greatly reduce the assignable requirement:
> 
> http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-defects.html#276

Thank you Howard, but that will not make it. What I really need are 
random access containers with neither assignable nor copy-constructible 
requirements. Those are kinds of things needed in GUI development.

This proposal speaks only about assignable and only in context of 
node-based containers. That is useless, at least for me.

> I hope to further reduce the need for both CopyConstructible and 
> CopyAssignable both in containers, and in algorithms, with the 
> introduction of the rvalue reference:
> 
> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1771.html

Sure, I know, perhaps you remember that I am big fan.

BTW, have you finally realised how important are composition rules there?

Well, even without them this will help a lot, but with them C++ would be 
paradise.

Mirek

P.S.: If I remember well, last time we were discussing, you was 
suggesting that with (implicit) destructive copy it is impossible to 
implement merge sort. Well, I was not sure at the moment, but now I know 
that it is possible and trivial... In general, default destructive copy 
has only a little impact on algorithms library.

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Russell Hind wrote:

> I took up to a year for me to be confident enough with bind to decide it
> may be better than a hand-written loop.  But there are cases where it
> just looks complicated.  e.g. some of our data structures have a
> 'get_name' member function with returns a string.  We want to find which
> one has a particular name and the bind expression comes out to be:
>
> find_if(m_ExistingExcipients.begin(), m_ExistingExcipients.end(),
>          bind(equal_to<string>(), bind(&Excipient_c::GetDescription,
> _1), Name));

[...]

> Sorting them by description gives us
>
> std::sort(Excipients.begin(), Excipients.end(),
>            bind(less<string>(), bind(&Excipient_c::GetDescription, _1),
>                                 bind(&Excipient_c::GetDescription, _2)));

The current CVS version of boost::bind (soon to be released) supports

bind(&Excipient_c::GetDescription, _1) == Name

and

bind(&Excipient_c::GetDescription, _1) <
bind(&Excipient_c::GetDescription, _2)


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Marco Manfredini wrote:
> i.e. in ~8.8 Milliones lines of open source c++ code, less
> than 200 uses of comparably important constructs from
> <algorithm> and <functional>.  Not sure if that means
> something...

Yes. It indicates that the comments describing said uses of
find_if, bind1st, etc are typically about 40,000 lines long.

:-)


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Scott Meyers wrote:

> >From my perspective, there are two questions here.  One is, as I said,
> whether it is practical to replace many manual loops with algorithm calls.

Algorithm functions are very specific in a way that they work with
pointers that represent sequences.
Most loops are not of that kind, rather they use counters and/or
indexing variables.
Simple loop like:

int a[10];
 for(size_t i = 0; i<10;++i)a[i]=i;

can be efficiently translated to algorithm function.

struct Nplus{
Nplus(int n=0):n_(n){}
int operator()(){ return n_++; }
int n_;
};

generate(a,a+10,Nplus());

but, people are not used to think that way and with more complex loops
there is really a question if such translation would be pragmatic.
Regarding simulations of lambda functions and bind, I never do this,
rather, if necessary, provide specific overloads.

> The other is where people can find open-source examples of real systems
> that actually do it.

Generally people try to avoid templates, but that was some time ago.
Now, since compilers are improving I don't see reason to avoid
them in the future.

Greetings, Bane.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

>>From my perspective, there are two questions here.  One is, as I said,
> whether it is practical to replace many manual loops with algorithm calls.


I work on a ~50K line program in the electronics/FPGA field which uses 
the Qt library. Initially when I started working on the project I tried 
to use algorithms with function objects, but eventually moved away from 
them. I got sick of passing class member variables into the function 
object constructors, it just seemed uneccessary when you can just write 
the loop in the function directly. With Qt4.0 they have a foreach macro 
which I will start using when I port from Qt 3.3 to Qt 4. Essentially 
our application seems to be moving further and further away from the 
standard library, we don't even use std::string because QString makes 
i18n so easy. I think I first started trying to use the standard 
algorithms because I thought they would be a cool challenge, but the 
novelty soon wore off. I do think they are seriously too different from 
other major programming languages like C# and Java and this is important 
when hiring people, especially in my area of New Zealand, where the 
local university typically only teaches Java, C and a little Haskell 
through the course of a computer science degree.

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
>>> From my perspective, there are two questions here.  One is, as I said,
>> whether it is practical to replace many manual loops with algorithm
>> calls.
>
> IME, no.
>
> IME, and I know most people will disagree with me, the most effective
> solution is to avoid node base container and iterators and use indicies
> everywhere.

Good luck for having an indexed map. :-)

> Makes coding trivial, short and unlikely to be prone to hard to catch
> errors like silently invalidated iterators or iterators going out of
> range (ok, I know that some STL implementation provide runtime checks,
> but thing is that these kinds of error are much less likely to happen
> with indicies).

In what way are indices more likely to stay valid than iterators?  I mean if 
I have index #10 tored away for a vector, and then I remove stuff and have 
only 5 elements, my index becomes rogue just the same way an iterator would. 
I just do not see (or believe) that it is any less likely with indices than 
iterators.

> I believe that all that "for_each" madness was cause by two factors:
>
> - it looks "cool" and "advanced"

Well, here we disagree again.  I think that Claudia Schiffer looks cool, and 
tablet PCs look advanced. :-)))

Of course, it *is* motivation for some people to "just do it" and make code 
just to show off.  But IMHO most successful C++ programmers (still being 
programmers and alive ones ;-) ) do not do that.  One uses a language (or 
library) feature, because the design is expressed best with that solution.

> - it is annoying to create for loop for STL containers as iterators
> syntax is usually way too ugly
>
> for(std::vector<std::string>::iterator it = c.begin(); it != c.end();
> it++)

Indeed it is!  It is so annoying, that I had to work through already 3 nites 
in this (well, the previous) month to fix code like the above.  Why?  In 
that code there is one error, one serious design flow and one possible 
performance hit!  In a single line of code!

What does that tell us?  It is *indeed* advicable to use STL algorithms, 
instead of badly written loops.

1.) ++it and *not* it++.  For gods sake: Mean what you say, and say what you 
mean!

2.) std::vector<std::string>::iterator is something, for which I flunk 
pupils.  Over 40 ones.  Is it really necessary to repeate that 
std::vector<std::string> in 254 places of the code?  How about having a

typedef std::vector<std::string> str_table_t;

inside that class having that very container as a member?  So if tomorrow I 
decide to use list, or deque, or my own home grown STL-compatible 
fixed-capacity-array class all I need to do is to change that one line of 
code.  (As long as the new container fits the bill, but it does, that is why 
I chose it.)

3.) Unless the container itself is being changed inside the loop, the end() 
iterator is better be saved away into a const variable before the loop.

4.) NITPICKING.  Use variable names which tell what the thing *represents*, 
not how it is implemented.  c might be A OK for the speed of light, maybe 
for a character (but conventionally that is ch), but it is a very bad name 
for a vector of strings.  Since that is a "symbol table", or a "name list", 
or a "state list" or... Anything, but a container and a c.  At least in 
application code, which most of us is supposed to write.

> I guess this is something that nobody really likes... (I think that at
> the moment "auto" extension is introduced, all that "for_each" stuff
> will be silently forgotten in year or two).

I strongly doubt that.  I think that as soon as lamba expressions (and 
possibly closure like things) get supported by the core language better, we 
will see a lot more use of algorithms than we do today.

Why do I say that algorithms are better than hand written loops?  Reason #1 
is the 3 mistakes made in that loop above.  Reason #2 is that I am 
*extremely* lazy.  I am ready to waste my time with internet chats, with 
many silly things...  But I do not like spending even 2 seconds (while I am 
in a thinking process in understanding code) to find out what that loop 
stands for.  std::for_each already gives a hint on that-

> As for me, I prefer easy to maintain variant
>
> for(int i = 0; i < c.GetCount(); i++)

God save me!  ++i.  Say what you mean, mean what you say!  Pretty please! 
:-)

GetCount()?  I like my Brasilian coffee, grown under a different Sun.  Why 
on Earth GetCount()?  How does that name is better than size()?  Not 
counting the absolutely unneded 3 letters (get), the unconventional casing 
(most people are used to see Types started with uppercase letters but not 
functions) and the fact that it suggests that it *counts* something, while 
it does not. :-(((

I must be very slow now, it is late here.  But I have failed to grasp how 
two, badly written, for loops are proving that algorithms are bad.

-- 
WW aka Attila
:::
Historically speaking, the presence of wheels in Unix has never precluded
their reinvention. - Larry Wall 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> > Changes to list, set, multiset, map, multimap are already in the C++0X
> > working paper which greatly reduce the assignable requirement:
> >
> > http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-defects.html#276
>
> Thank you Howard, but that will not make it. What I really need are
> random access containers with neither assignable nor copy-constructible
> requirements. Those are kinds of things needed in GUI development.

I'm curious; what kind of non-assignable, non-copy-constructible
objects do you need to put into containers in GUI development? Or did I
misunderstand something?

Bob


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

wij@seed.net.tw <wij@seed.net.tw> wrote:
> [...]
> Let's say, If I already know how to program in C++, why
> bother spending longer time to learn STL [...]

  Er...because you do /not/ know how to program
  in C++ unless you learned how to use the STL?

> [...]


 Schobi

-- 
SpamTrap@gmx.de is never read
I'm Schobi at suespammers dot org

"Coming back to where you started is not the same as never leaving"
 Terry Pratchett



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> function that could be 30 lines long with just a few if-elses and 5
> calls to STL algorithms becomes 200 lines, then it is not easier to
> read.

If only that was true. I guess that very often, almost in any 
non-trivial case, hand coded loops are as long or even shorter than 
algorithms. Which in the end is what was pointed out in original post.

> Perhaps it would be a good idea that people post *concrete examples*
> of a given situation, and their choice for that particular situation;
> for instance, if I have a list of Packets to be transmitted, and each
> Packet has a time tag (the time at which it was dispatched to be sent),
> and I'm looking for the oldest, then I'd definitely use the STL:
> 
> class cmp_timetag
> {
> public:
>      bool operator() (const Packet & p1, const Packet & p2) const
>      {
>          return p1.sent_at() < p2.sent_at();
>      }
> };
> 
> min_element (transmit_queue.begin(),
>               transmit_queue.end(),
>               cmp_timetag());
> 
> 
> If I need to count how many strings are longer than a given string,
> I'd definitely use the STL:
> 
> class longer_than
> {
>      const string & s;
> public:
>      longer_than (const string & s) : s(s) {}
> 
>      bool operator() (const string & item) const
>      {
>          return item.length() > s.length();
>      }
> };
> 
> count_if (names.begin(), names.end(), longer_than("Carlos"));

Well, I guess you have just proved that my point above is correct...

Hand coded, you would avoid using two predicate classes and reduced 
above code to much less lines.

(Also, a little bit off topic, I would store length of s rather than the 
reference to s in longer_than, but that is just me :)

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

>>IME, and I know most people will disagree with me, the most effective
>>solution is to avoid node base container and iterators and use indicies
>>everywhere.
> 
> 
> Good luck for having an indexed map. :-)

Like this:

http://upp.sourceforge.net/src$ArrayMap$en-us.html

?

:)

(And yes, it is faster than std::map for average case, before you start 
asking).

> In what way are indices more likely to stay valid than iterators?  I mean if 
> I have index #10 tored away for a vector, and then I remove stuff and have 
> only 5 elements, my index becomes rogue just the same way an iterator would. 
> I just do not see (or believe) that it is any less likely with indices than 
> iterators.

Well, loops based on indicies have usually i < c.GetCount() (or i < 
size() if you insist) condition. That rules out most of problems.

Usually, the body of loop performs some single action. This action might 
modify the container. If this accidentally happens with iterators and 
say vector, you are out of luck (iterators get invalidated, or get out 
of range and you are testing just equality to End).

With indicies it usually works as expected out of box without trouble.

If you want example, go several news threads back and see "Vector 
Iterator headache". And I guess it is at least one post per week that 
deals with exactly the same problem.

And, BTW, performance wise, there is no difference either. Compiler will 
emit similary effective code regardless you are using indicies or iterators.

> Of course, it *is* motivation for some people to "just do it" and make code 
> just to show off.  But IMHO most successful C++ programmers (still being 
> programmers and alive ones ;-) ) do not do that.  One uses a language (or 
> library) feature, because the design is expressed best with that solution.

Well, why it is "recommended style" then? That is I think OP's issue.

> 2.) std::vector<std::string>::iterator is something, for which I flunk 
> pupils.  Over 40 ones.  Is it really necessary to repeate that 
> std::vector<std::string> in 254 places of the code?  How about having a
> 
> typedef std::vector<std::string> str_table_t;
> 
> inside that class having that very container as a member?


Great. Now you had to introduce another type just to keep your broken 
machine going. Adding entities to code just to make your loops run does 
nothing than increase your code complexity.

> So if tomorrow I 
> decide to use list, or deque

So what? How str_table_t is going to help you? Do you plan to replace 
ALL occurences of std::vector<std::string> in your code with list or 
deque? Or do you plan to introduce

str_table_for_my_Foo_class_implementation_t

like things?

Anyway, for me this is much more simple. If I want to replace one of my 
vector-like container with another, I just do that. They have similar 
interface and no additional types needed to write code with them (thanks 
to indicies).

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <dc919p$94s$1@phys-news1.kolumbus.fi>, White Wolf 
<wolof@freemail.hu> writes
>In what way are indices more likely to stay valid than iterators?  I mean if
>I have index #10 tored away for a vector, and then I remove stuff and have
>only 5 elements, my index becomes rogue just the same way an iterator would.
>I just do not see (or believe) that it is any less likely with indices than
>iterators.

Very simply:

if(index < v.size()) v[index] ...

remains valid. IOWs the precondition for v[n] to be valid if v is a 
std::vector<type> is that v currently has at least n elements. There is 
an extra precondition for *v_iter to be valid; that the vector has not 
been moved since the value in v_iter was obtained. That pre-condition is 
not even checkable in the general case.


-- 
Francis Glassborow      ACCU
Author of 'You Can Do It!' see http://www.spellen.org/youcandoit
For project ideas and contributions: http://www.spellen.org/youcandoit/projects


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Scott Meyers" <Usenet@aristeia.com> wrote in message 
news:MPG.1d50a59dd15264319897e0@news.hevanet.com...
....
>>From my perspective, there are two questions here.  One is, as I said,
> whether it is practical to replace many manual loops with algorithm calls.
> The other is where people can find open-source examples of real systems
> that actually do it.

Can only answer the first question from my personal experience. In our code 
base there are plenty of manual loops and many less algorithms. When I tried 
using algorithms intensively in one library I found out two problems with 
that.

Firstly, algorithms often require some sort of functor object. As a result 
implementation was littered with hundreds of small classes. And those 
classes may be separated by many lines of code from the place where they are 
used, because the standard doesn't allow template instantiation on local 
classes.

Secondly, since the only way to pass the local context is to copy variables 
(or pointers/references) there is a lot of constructors that are copying 
stuff and it's very inflexible and a pain to extend. The real code is often 
designed by try and error process. It's not unusual to have several attempts 
before settling on a certain algorithm and functor. Imagine how much 
unnecessary typing it involves. So you can find that most algorithms that 
are frequently used are those that do not require creating a functor. Manual 
loops don't have these deficiencies: the code that does something is 
localized right at the place of where the action is, and the local context 
is easily available. Now, lambda and binders have some of same advantages. 
But they also have problems. Compilation errors can be baffling, and it's 
practically impossible to debug nontrivial compositions.

- gene 


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Bob Bell wrote:
> Mirek Fidler wrote:
> 
>>>Changes to list, set, multiset, map, multimap are already in the C++0X
>>>working paper which greatly reduce the assignable requirement:
>>>
>>>http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-defects.html#276
>>
>>Thank you Howard, but that will not make it. What I really need are
>>random access containers with neither assignable nor copy-constructible
>>requirements. Those are kinds of things needed in GUI development.
> 
> 
> I'm curious; what kind of non-assignable, non-copy-constructible
> objects do you need to put into containers in GUI development? Or did I
> misunderstand something?

Well, namely, widgets.

Like Array<Button>.

In general development, things like file streams are sometimes needed as 
well.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

chris jefferson wrote:
> I mean, considering the body of for_each is basically:
>       for ( ; __first != __last; ++__first)
>         __f(*__first);
>

Although I think that 'for_each()' is not really the most intersting
algorithm in STL, it can actually be implemented quite differently
and take advantage of some optimizations, especially if the function
used is pretty simple. For example, 'for_each()' can do loop unrolling
if the sequence provides random access.
-- 
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> P.S.: I hope that it is not quite off-topic: as an alternative, I dared
> to develop, maintain and use this:
>
> http://upp.sourceforge.net/srcdoc$NTL$en-us.html
>
> the main difference is the lack of "copy-constructible" and "assignable"
> requirements, I believe that it immediately cuts off at least 70% of
> problems that I have encountered in STL.

The really major difference is that you are addressing something
entirely different than STL does! You are not at all addressing
algorithms which is the primary goal of STL. You have some containers
as has STL but the containers for STL, albeit useful, are mostly
there as examples of sequences. I'd bet that your containers could
also be used with STL algorithms. The basic idea of STL is that
algorithms are written once and can then be applied to all data
structures having a reasonable representation (e.g. can be traversed
in some form or have some form of random access, depending on the
algorithm).
-- 
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <3kqejmFvmmlqU1@individual.net>, Mirek Fidler <cxl@volny.cz>
wrote:

> BTW, have you finally realised how important are composition rules there?

Nope, but I'm willing to be educated.

-Howard

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> In what way are indices more likely to stay valid than iterators?

If you have a vector that is full to capacity and you do a extra push_back()
that normally invalidates any iterators as the underlying memory is
reallocated. An index to an element before the push_back()'ed element is
still valid.

That is a big difference.

Stephen Howe



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <1122507091.134881.5960@g14g2000cwa.googlegroups.com>,
  "Bob Bell" <belvis@pacbell.net> wrote:

> Mirek Fidler wrote:
> > > Changes to list, set, multiset, map, multimap are already in the C++0X
> > > working paper which greatly reduce the assignable requirement:
> > >
> > > http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-defects.html#276
> >
> > Thank you Howard, but that will not make it. What I really need are
> > random access containers with neither assignable nor copy-constructible
> > requirements. Those are kinds of things needed in GUI development.
>
> I'm curious; what kind of non-assignable, non-copy-constructible
> objects do you need to put into containers in GUI development? Or did I
> misunderstand something?

Well, not specific to GUI development, but certainly applicable there,
I'm thinking vector<unique_ptr<T>> might be extremely useful.
unique_ptr is like auto_ptr, but not copyable, only movable.  The T
might be a windowing abstract base class that is also not copyable, and
perhaps not even movable.

Because unique_ptr is not copyable, it is safe to use within generic
code such as containers and algorithms.  If the generic code attempts a
copy, it will fail at compile time.  However because unique_ptr is
movable, and because much generic code only requires moving instead of
copying, significant functionality with existing (but reimplemented)
standard containers and algorithms can be made available.

vector<shared_ptr<T>> is also useful in this context.  But
vector<unique_ptr<T>> is significantly faster, having the exact same
overhead as vector<T*>.  If you need the semantics of shared ownership,
shared_ptr is the tool you need.  However if you need the semantics of
unique (unshared) ownership, then unique_ptr can enforce that
requirement at compile time.  And yet you can still do things like:

vector<unique_ptr<int> > v1, v2;
.....
remove_copy(make_move_iterator(v1.begin()), make_move_iterator(v1.end()),
             back_inserter(v2), unique_ptr<int>());

This call to remove_copy moves all non-null unique_ptr's from v1 to v2.

(quoted from
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1771.html )

Other candidates of non-copyable but movable objects that might benefit
from being in a container include streams and locks.

-Howard

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"White Wolf" <wolof@freemail.hu> writes:

| In what way are indices more likely to stay valid than iterators?  I mean if
| I have index #10 tored away for a vector, and then I remove stuff and have
| only 5 elements, my index becomes rogue just the same way an iterator would.
| I just do not see (or believe) that it is any less likely with indices than
| iterators.

push_back is most likely to invalid previous iterators than shrinking
a vector.

-- 
                                                        Gabriel Dos Reis
                                            gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler <cxl@volny.cz> writes:

| > Changes to list, set, multiset, map, multimap are already in the C++0X
| > working paper which greatly reduce the assignable requirement:
| >
| > http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-defects.html#276
|
| Thank you Howard, but that will not make it. What I really need are
| random access containers with neither assignable nor copy-constructible
| requirements.

I can say

      ofstream dumpfiles[max_dump_index];

or

     new ofstream[max_dump_index];

yet, not

     vector<ofstream> dumpfiles;

:-(

-- 
                                                        Gabriel Dos Reis
                                            gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Bob Bell" <belvis@pacbell.net> writes:

| Mirek Fidler wrote:
| > > Changes to list, set, multiset, map, multimap are already in the C++0X
| > > working paper which greatly reduce the assignable requirement:
| > >
| > > http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-defects.html#276
| >
| > Thank you Howard, but that will not make it. What I really need are
| > random access containers with neither assignable nor copy-constructible
| > requirements. Those are kinds of things needed in GUI development.
|
| I'm curious; what kind of non-assignable, non-copy-constructible
| objects do you need to put into containers in GUI development? Or did I
| misunderstand something?

Maybe not in a GUI development, but is there a reason why I can say

     ofstream dumpfiles[max];

or

    new ofstream[max];

and not

     vector<ofstream> dumpfiles(max);

?

-- 
                                                        Gabriel Dos Reis
                                            gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Peter Dimov wrote:
>
> The current CVS version of boost::bind (soon to be released) supports
>
> bind(&Excipient_c::GetDescription, _1) == Name
>
> and
>
> bind(&Excipient_c::GetDescription, _1) <
> bind(&Excipient_c::GetDescription, _2)
>

Thanks, I didn't realise this.  That will help a lot.

Just out of interest, why isn't this sort of thing mentioned in the
latest news for 1.33.0
(http://cvs.sourceforge.net/viewcvs.py/*checkout*/boost/boost/index.htm?rev=1.230)?
   My general impression is that libraries that aren't mentioned on
there, haven't changed since the previous release and therefore I don't
see an easy way for end-users to find out about updates to libraries
they already use which is a shame IMHO as we will end-up using the
library in the same way as we have been for the last year or so.

Cheers

Russell

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

M Jared Finder wrote:
>
> Just to be pedantic, the second example is the equivalent of
> std::find_if, not of std::sort.  std::sort can not be written
> efficiently  using BOOST_FOREACH, as it requires mutating the underlying
> sequence in multiple passes.
>

I realise that the BOOST_FOREACH example should have gone before the
std::sort, but thats just the way they popped in to my head (don't use
BOOST_FOREACH yet in production stuff as it isn't in an official boost
release so it isn't the first thing on my mind when doing loops yet).

There certinaly are some issues with BOOST_FOREACH such as not being
able to modify the container in such a way, but as mentioned, I think
the more limiting factor is that you just don't get access to the actual
iterator which you may want to store for later use in another algorithm
or such so this will limit its use.

The only other slight issue I have with it is that if you want to modify
the values in-place, you have to remember to declare the loop variable
as a reference.  I have a feeling this might be a common mistake such as

vector<int> v;
BOOST_FOREACH(int i, v)
{
    i *= 2;
}

will not do anything useful.

Cheers

Russell

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

>>BTW, have you finally realised how important are composition rules there?
> 
> 
> Nope, but I'm willing to be educated.

Well, nothing too complicated, it is just if I remember well, last 
version of proposal gave up on composition rules.

I mean, if I have

struct Foo {
	TypeWithRValueCopy  a;
	TypeWithNormalCopy  b;
};

Here I expect compiler to generate default r-value copy constructor and 
assignment operator for Foo.

This is actually similar today when compiler generates const T& and T& 
copy constructors and assignment operator. When there is any subtype 
with non-const copy constructor, copy constructor for whole class is 
non-const as well, if there are const only, whole class copy constructor 
is const too.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Gabriel Dos Reis wrote:
> Mirek Fidler <cxl@volny.cz> writes:
> 
> | > Changes to list, set, multiset, map, multimap are already in the C++0X
> | > working paper which greatly reduce the assignable requirement:
> | >
> | > http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-defects.html#276
> |
> | Thank you Howard, but that will not make it. What I really need are
> | random access containers with neither assignable nor copy-constructible
> | requirements.
> 
> I can say
> 
>       ofstream dumpfiles[max_dump_index];
> 
> or
> 
>      new ofstream[max_dump_index];
> 
> yet, not
> 
>      vector<ofstream> dumpfiles;

Well, I am saying

Array<oftream> dumpfiles;

or

ArrayMap<String, ofstream> dumpfiles_map;

for more than 7 years now.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Gene Bushuyev wrote:

[...]

> Secondly, since the only way to pass the local context is to copy variables
> (or pointers/references) there is a lot of constructors that are copying
> stuff and it's very inflexible and a pain to extend. The real code is often
> designed by try and error process. [...]

One approach that works for me is to use a stateless function (object)
combined with boost::bind to hold local state.

> Now, lambda and binders have some of same advantages.
> But they also have problems. Compilation errors can be baffling, and it's
> practically impossible to debug nontrivial compositions.

A nontrivial composition is usually an indication that one should go
with a loop or a custom algorithm instead. It's not a global either/or
decision.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

>>the main difference is the lack of "copy-constructible" and "assignable"
>>requirements, I believe that it immediately cuts off at least 70% of
>>problems that I have encountered in STL.
> 
> 
> The really major difference is that you are addressing something
> entirely different than STL does! You are not at all addressing
> algorithms which is the primary goal of STL.  You have some containers
> as has STL but the containers for STL, albeit useful, are mostly
> there as examples of sequences.

Dietmar, you was saying this before, but I think that this is not true.

STL is a library of containers and algorithms. Containers in STL are not 
"examples of sequences".

> I'd bet that your containers could
> also be used with STL algorithms.

Yes, as long as types stored satisfy general STL requirements, which 
often is not true. That is why I had to develop some algorithms anyway, 
as they are guaranteed to work with types with different set of 
requirements.

> The basic idea of STL is that
> algorithms are written once and can then be applied to all data
> structures having a reasonable representation
> (e.g. can be traversed
> in some form or have some form of random access, depending on the
> algorithm).

This might be true for some non-mutating algorithms.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Howard Hinnant wrote:
> In article <1122507091.134881.5960@g14g2000cwa.googlegroups.com>,
>   "Bob Bell" <belvis@pacbell.net> wrote:
> 
> 
>>Mirek Fidler wrote:
>>
>>>>Changes to list, set, multiset, map, multimap are already in the C++0X
>>>>working paper which greatly reduce the assignable requirement:
>>>>
>>>>http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-defects.html#276
>>>
>>>Thank you Howard, but that will not make it. What I really need are
>>>random access containers with neither assignable nor copy-constructible
>>>requirements. Those are kinds of things needed in GUI development.
>>
>>I'm curious; what kind of non-assignable, non-copy-constructible
>>objects do you need to put into containers in GUI development? Or did I
>>misunderstand something?
> 
> 
> Well, not specific to GUI development, but certainly applicable there,
> I'm thinking vector<unique_ptr<T>> might be extremely useful.

Like this

http://upp.sourceforge.net/src$Array$en-us.html

?

Yes, that is useful, very useful.

> Other candidates of non-copyable but movable objects that might benefit
> from being in a container include streams and locks.

Exactly. What you dream of I am using :)

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler <cxl@volny.cz> writes:

| > I can say
| > 
| >       ofstream dumpfiles[max_dump_index];
| > 
| > or
| > 
| >      new ofstream[max_dump_index];
| > 
| > yet, not
| > 
| >      vector<ofstream> dumpfiles;
| 
| Well, I am saying
| 
| Array<oftream> dumpfiles;
| 
| or
| 
| ArrayMap<String, ofstream> dumpfiles_map;
| 
| for more than 7 years now.

I know how to say

   MyArray<ofstream> dumpfiles(max_dump_index);

for more than 7 years.  That wasn't the point.


-- 
                                                       Gabriel Dos Reis 
                                           gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Howard Hinnant <hinnant@metrowerks.com> writes:

| In article <1122507091.134881.5960@g14g2000cwa.googlegroups.com>,
|   "Bob Bell" <belvis@pacbell.net> wrote:
| 
| > Mirek Fidler wrote:
| > > > Changes to list, set, multiset, map, multimap are already in the C++0X
| > > > working paper which greatly reduce the assignable requirement:
| > > >
| > > > http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-defects.html#276
| > >
| > > Thank you Howard, but that will not make it. What I really need are
| > > random access containers with neither assignable nor copy-constructible
| > > requirements. Those are kinds of things needed in GUI development.
| >
| > I'm curious; what kind of non-assignable, non-copy-constructible
| > objects do you need to put into containers in GUI development? Or did I
| > misunderstand something?
| 
| Well, not specific to GUI development, but certainly applicable there,
| I'm thinking vector<unique_ptr<T>> might be extremely useful.

so that we have to allocate a vector of pointers just to hold data that
never move or are never copied?  That is a simple basic thing.  I hope
that is not all we can do for basic things.

| unique_ptr is like auto_ptr, but not copyable, only movable.  The T
| might be a windowing abstract base class that is also not copyable, and
| perhaps not even movable.

then why can't I put the data directly in there, instead of allocating
them somewhere else and then allocate pointers for them and finally
put the pointers in the allocated non-copyable and only-moveable storage?

| Because unique_ptr is not copyable, it is safe to use within generic
| code such as containers and algorithms.  If the generic code attempts a
| copy, it will fail at compile time. 

that is good, but what if I do not do any of those algorithms?  I
simply want to put the data in a sequence, is there anything for me?

| However because unique_ptr is
| movable, and because much generic code only requires moving instead of
| copying, significant functionality with existing (but reimplemented)
| standard containers and algorithms can be made available.
| 
| vector<shared_ptr<T>> is also useful in this context.  But

OK; now, I'll start the "Down with smart pointers!" movement :-)

-- 
                                                       Gabriel Dos Reis 
                                           gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler <cxl@volny.cz> writes:

| Containers in STL are not "examples of sequences".

What are they?

-- 
                                                       Gabriel Dos Reis 
                                           gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <3ks0u6FvaveeU1@individual.net>, Mirek Fidler <cxl@volny.cz> 
wrote:

> > I'd bet that your containers could
> > also be used with STL algorithms.
> 
> Yes, as long as types stored satisfy general STL requirements, which 
> often is not true. That is why I had to develop some algorithms anyway, 
> as they are guaranteed to work with types with different set of 
> requirements.
> 
> > The basic idea of STL is that
> > algorithms are written once and can then be applied to all data
> > structures having a reasonable representation
> > (e.g. can be traversed
> > in some form or have some form of random access, depending on the
> > algorithm).
> 
> This might be true for some non-mutating algorithms.

The rvalue reference proposal - library recommendations ( 
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1771.html ) 
addresses algorithms as well as the containers.  Check out the number of 
mutating algorithms listed in there that do not require 
CopyConstructible or CopyAssignable.  They include:

remove
remove_if
unique
reverse
rotate
random_shuffle
partition
stable_partition
sort
stable_sort
partial_sort
nth_element
inplace_merge
the heap algorithms
the permutation algorithms

This proposal is backed up by a full working implementation.

-Howard

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Hendrik Schober wrote:
> Er...because you do /not/ know how to program
> in C++ unless you learned how to use the STL?

1. It is invalid to conclude that way beside untrue premise.
2. Must I use STL, or my program is less C++?
3. STL is good for many classes 'that fit its requirements'.
   (many classes I am currently using simply do not fit)


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On Wed, 27 Jul 2005 19:19:47 -0400, White Wolf wrote:


>> Makes coding trivial, short and unlikely to be prone to hard to catch
>> errors like silently invalidated iterators or iterators going out of
>> range (ok, I know that some STL implementation provide runtime checks,
>> but thing is that these kinds of error are much less likely to happen
>> with indicies).
> 
> In what way are indices more likely to stay valid than iterators?  I mean if 
> I have index #10 tored away for a vector, and then I remove stuff and have 
> only 5 elements, my index becomes rogue just the same way an iterator would. 
> I just do not see (or believe) that it is any less likely with indices than 
> iterators.

If you append stuff to a std::vector, iterators become invalid, indices do
not. 

[...]
 

> 1.) ++it and *not* it++.  For gods sake: Mean what you say, and say what
> you mean!

I don't quite get what you mean here...


> 3.) Unless the container itself is being changed inside the loop, the end() 
> iterator is better be saved away into a const variable before the loop.

This is no longer true:

Here's a translation unit:

#include <list>
using namespace std;

list<int> stuff;
typedef list<int>::iterator it;

int foo()
{
	int a=0;
	for(it i=stuff.begin(); i != stuff.end(); ++i)
		a += *i;
	return a;
}

int baz()
{
	int a=0;
	for(it i=stuff.begin(); i != stuff.end(); i++)
		a += *i;
	return a;
}
int bar()
{
	int a=0;
	it end = stuff.end();
	for(it i=stuff.begin(); i != end; ++i)
		a += *i;
	return a;
}


Here's the assembly output from g++ -O3 (g++ 4.0.0, similar results for
3.3.5)

 _Z3foov:                      | _Z3barv:
 .LFB391:                      | .LFB392:
         movl    stuff, %edx   | 	movl	stuff, %edx
         xorl    %eax, %eax    | 	xorl	%eax, %eax
         pushl   %ebp          | 	pushl	%ebp
 .LCFI2:                       | .LCFI0:
         movl    %esp, %ebp    | 	movl	%esp, %ebp
 .LCFI3:                       | .LCFI1:
         cmpl    $stuff, %edx  | 	cmpl	$stuff, %edx
         je      .L12          | 	je	.L4
         .p2align 4,,15        | 	.p2align 4,,15
 .L13:                         | .L5:
         movl    8(%edx), %ecx | 	movl	8(%edx), %ecx
         movl    (%edx), %edx  | 	movl	(%edx), %edx
         addl    %ecx, %eax    | 	addl	%ecx, %eax
         cmpl    $stuff, %edx  | 	cmpl	$stuff, %edx
         jne     .L13          | 	jne	.L5
 .L12:                         | .L4:
         popl    %ebp          | 	popl	%ebp
         ret                   | 	ret

in both cases, the code is the same. In other words, there's no need in
this case to second-guess the compiler. In the case of baz, for g++ 4.0.0,
the code is the same again. For 3.3.5, it is different, and slower:


With g++ 3.3.4 (same code (ish) but a complete program) over 100 runs, we
get: 
++i
mean = 4.88
std  = 0.0277

i++
mean = 5.5227s
std  = 0.0289


In conclusion, on a more modern compiler, i++ and ++i have no performance
difference. On even slightly less modern compilers, there is no need to
use a temporary variable for .end(), since the compiler can figure it out
for itself.


-Ed


-- 
(You can't go wrong with psycho-rats.)       (er258)(@)(eng.cam)(.ac.uk)

/d{def}def/f{/Times findfont s scalefont setfont}d/s{10}d/r{roll}d f 5/m
{moveto}d -1 r 230 350 m 0 1 179{1 index show 88 rotate 4 mul 0 rmoveto}
for /s 15 d f pop 240 420 m 0 1 3 { 4 2 1 r sub -1 r show } for showpage


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler <cxl@volny.cz> wrote:
> [...]
> > for_each(clients.begin(), clients.end(), CancelAllSignatures());
> >
> > I think it's much better than a loop. It's obvious we will cancel all
> > signatures from all clients.
>
> Is it?

  Yes.

>        What CancelAllSignatures returns? Is it a functor?

  I couldn't care less. No matter /how/ this is
  implemented, by the first look at it, I (having
  /no/ experience at all in the application domain)
  see /what/ it will do.
  And it's certaonly easier to read than

    for( clients_t::iterator it=clients.begin(), it!=clients.end(); ++it )
      CancelAllSignatures();

  (I'd argue that
    for_each( clients, CancelAllSignatures() );
  would be even easier.)

> [...]
> Mirek

  Schobi

-- 
SpamTrap@gmx.de is never read
I'm Schobi at suespammers dot org

"Coming back to where you started is not the same as never leaving"
 Terry Pratchett



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Russell Hind" <rh_gmane@mac.com> wrote in message 
news:dca14e$ppo$1$8300dec7@news.demon.co.uk...
> Peter Dimov wrote:
>>
>> The current CVS version of boost::bind (soon to be released) supports
>>
>> bind(&Excipient_c::GetDescription, _1) == Name
>>
>> and
>>
>> bind(&Excipient_c::GetDescription, _1) <
>> bind(&Excipient_c::GetDescription, _2)
>
> Thanks, I didn't realise this.  That will help a lot.
>
> Just out of interest, why isn't this sort of thing mentioned in the
> latest news for 1.33.0
> (http://cvs.sourceforge.net/viewcvs.py/*checkout*/boost/boost/index.htm?rev=1.230)?
>   My general impression is that libraries that aren't mentioned on
> there, haven't changed since the previous release and therefore I don't
> see an easy way for end-users to find out about updates to libraries
> they already use which is a shame IMHO as we will end-up using the
> library in the same way as we have been for the last year or so.

Agreed. This is a quantum improvement in simplification/readability of user 
code! Awesome work Peter. Is this also part of TR1?

Thanks, Jeff 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
>>function that could be 30 lines long with just a few if-elses and 5
>>calls to STL algorithms becomes 200 lines, then it is not easier to
>>read.
> 
> 
> If only that was true. I guess that very often, almost in any 
> non-trivial case, hand coded loops are as long or even shorter than 
> algorithms. Which in the end is what was pointed out in original post.


Agreed -- but see below.


>>class cmp_timetag
>>{
>>public:
>>     bool operator() (const Packet & p1, const Packet & p2) const
>>     {
>>         return p1.sent_at() < p2.sent_at();
>>     }
>>};
>>
>>min_element (transmit_queue.begin(),
>>              transmit_queue.end(),
>>              cmp_timetag());
>>
>>
>>If I need to count how many strings are longer than a given string,
>>I'd definitely use the STL:
>>
>>class longer_than
>>{
>>     const string & s;
>>public:
>>     longer_than (const string & s) : s(s) {}
>>
>>     bool operator() (const string & item) const
>>     {
>>         return item.length() > s.length();
>>     }
>>};
>>
>>count_if (names.begin(), names.end(), longer_than("Carlos"));
> 
> 
> Well, I guess you have just proved that my point above is correct...
> 
> Hand coded, you would avoid using two predicate classes and reduced 
> above code to much less lines.

But what really matters is the length of the "main code" -- auxiliary
functions or classes don't count in terms of making the code more
readable.

What matters is that you see *this one line* in your "main code":

count_if (names.begin(), names.end(), longer_than("Carlos"));

And that, not only is a single line (vs. four lines, without counting
the curly braces, which I personally always spend one line for each
of them, so that would make *eight* lines), but it makes it immediately
obvious that you're counting how many strings are longer than my name.
Writing the class was effort well-spent, because it was spent in writing
cleaner code.

As a subtle plus, if you have to count the number of strings longer
than a given one three times, then:  1) the effort paid off, and now
the class plus the three lines will be shorter than three hand-coded
loops.  And b) there *is* the risk that the second time you will forget
to initialize the counter back to zero -- not writing code is less
error-prone than writing code!  :-)

> (Also, a little bit off topic, I would store length of s rather than the 
> reference to s in longer_than, but that is just me :)

Duh!!!  (to clarify: I'm telling "duh" to myself  :-)).  Yeah, that was
certainly a nice way to give more credibility to my argument, eh? :-)

Cheers,

Carlos
--

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> > do I need to repeat the same for(xx::interator i =
> > x.begin... over and

> It is problem of iterator syntax, not of 'for'.

> > Example:

> > for_each(clients.begin(), clients.end(), CancelAllSignatures());

> > I think it's much better than a loop. It's obvious we will
> > cancel all signatures from all clients.

> Is it? What CancelAllSignatures returns? Is it a functor?

Who cares?  That's the whole point.  Whatever it is, it cancels
all signatures.  He's given an otherwise opaque operation a
name, and that helps readability.

> > But it's good because the code reader doesn't need to know
> > the implementation detail.

> Impementation detail of what? for_each? Come on.

I think he meant the implementation detail of
CancelAllSignatures.

It's certainly not a killer argument, at least not when used
with for_each.  You can (and shoul) also write:

    for ( size_t i = 0 ; i < clients.size() ; ++ i ) {
        cancelAllSignatures( clients[ i ] ) ;
    }

Which is clearer?  I'd say that the for_each version has an
edge.  Only a slight one, but still an edge.

Other algorithms have a bigger edge.  If I see find or find_if,
I know that a linear search is being used.  If I see the loop,
it's not too hard to figure out, but I still have to figure it
out.  If I see lower_bound, I know that a binary search is
used.  That one takes even more time to figure out just seeing
the loop.

--
James Kanze                                           GABI Software
Conseils en informatique orient�e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> | > I can say
> | > 
> | >       ofstream dumpfiles[max_dump_index];
> | > 
> | > or
> | > 
> | >      new ofstream[max_dump_index];
> | > 
> | > yet, not
> | > 
> | >      vector<ofstream> dumpfiles;
> | 
> | Well, I am saying
> | 
> | Array<oftream> dumpfiles;
> | 
> | or
> | 
> | ArrayMap<String, ofstream> dumpfiles_map;
> | 
> | for more than 7 years now.
> 
> I know how to say
> 
>    MyArray<ofstream> dumpfiles(max_dump_index);
> 
> for more than 7 years.  That wasn't the point.

Well, maybe I got your point wrong, but please notice the absence of 
"max_dump_index" in my post...

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Howard Hinnant wrote:

> // Call f(first, first+k) for each reversible permutation of
> //  [first, last) taken k items at a time.
> //  "reversible" may be the wrong word here.  The
> //  intent is that the reverse of a permutation doesn't
> //  count as a permutation, and is thus not applied.

Maybe "for_each_permutation_or_reverse"...

> for_each_circular_permutation

....and "for_each_rotation"?


Martin

-- 
Quidquid latine dictum sit, altum viditur.

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> In conclusion, on a more modern compiler, i++ and ++i have no performance

Well, technically speaking, that is true in most cases, but OP is right 
that in generic iterator algorithm, using ++i is better, because in 
theory, ++ operators for some strange container iterators could be 
defined in a way that really makes i++ significantly slower.

In fact, using "i++" is bad habit from my C years, anyway as I gave up 
on iterators in generic code in favor of indicies, I do not care too 
much anymore.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Gabriel Dos Reis wrote:
> Mirek Fidler <cxl@volny.cz> writes:
> 
> | Containers in STL are not "examples of sequences".
> 
> What are they?
> 
Ehm, pardon my english, perhaps I got bad context.

What I wanted to say is that I think that Dietmar thinks that STL 
containers are unimportant and are just examples for you how to build 
your own.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <m3ek9jyrbs.fsf@uniton.integrable-solutions.net>,
 Gabriel Dos Reis <gdr@integrable-solutions.net> wrote:

> | Well, not specific to GUI development, but certainly applicable there,
> | I'm thinking vector<unique_ptr<T>> might be extremely useful.
> 
> so that we have to allocate a vector of pointers just to hold data that
> never move or are never copied?  That is a simple basic thing.  I hope
> that is not all we can do for basic things.
> 
> | unique_ptr is like auto_ptr, but not copyable, only movable.  The T
> | might be a windowing abstract base class that is also not copyable, and
> | perhaps not even movable.
> 
> then why can't I put the data directly in there, instead of allocating
> them somewhere else and then allocate pointers for them and finally
> put the pointers in the allocated non-copyable and only-moveable storage?

You can (assuming proposal acceptance) put your movable but non-copyable 
data directly into containers.  No problem.  I was specifically 
addressing the heterogeneous container idiom where a pointer to a base 
class is used (with varying derived types at run time).  Here you have 
to use a pointer or reference to prevent slicing.

Of course, container<base&> is an interesting concept in its own right.  
I have often thought about that, but haven't developed or proposed it.  
I'm sure it is an interesting and solvable problem.  I'm not sure how 
useful it would be in the field.  But I have overwhelming evidence that 
containers of (smart) pointers to base classes are widely used.

> | Because unique_ptr is not copyable, it is safe to use within generic
> | code such as containers and algorithms.  If the generic code attempts a
> | copy, it will fail at compile time. 
> 
> that is good, but what if I do not do any of those algorithms?  I
> simply want to put the data in a sequence, is there anything for me?

Yes, no problem putting your move-only types directly into containers.

> | However because unique_ptr is
> | movable, and because much generic code only requires moving instead of
> | copying, significant functionality with existing (but reimplemented)
> | standard containers and algorithms can be made available.
> | 
> | vector<shared_ptr<T>> is also useful in this context.  But
> 
> OK; now, I'll start the "Down with smart pointers!" movement :-)

No problem, your container<T*> will still work tomorrow as well as it 
works today. :-)  And if you aren't concerned about slicing, your 
container<T> capability and performance is about to dramatically 
increase.

-Howard

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> What matters is that you see *this one line* in your "main code":
> 
> count_if (names.begin(), names.end(), longer_than("Carlos"));
> 
> And that, not only is a single line (vs. four lines, without counting
> the curly braces, which I personally always spend one line for each
> of them, so that would make *eight* lines), but it makes it immediately
> obvious that you're counting how many strings are longer than my name.

Well, but only as long as you know what 'longer_than' does. Otherwise it 
will mean studying code at completely unrelated place and remembering 
meaning of another symbol.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Gabriel Dos Reis wrote:

> "White Wolf" <wolof@freemail.hu> writes:
> 
> | In what way are indices more likely to stay valid than iterators?  I
> | mean if I have index #10 tored away for a vector, and then I remove
> | stuff and have only 5 elements, my index becomes rogue just the same way
> | an iterator would. I just do not see (or believe) that it is any less
> | likely with indices than iterators.
> 
> push_back is most likely to invalid previous iterators than shrinking
> a vector.

Actually, it is easy to create a container similar to vector which
does not suffer from invalidating iterators while mutating the
contents: the iterators would simply store a pointer to the container
and an index. In fact, you can even take 'std::vector' for this but
use different than the normal iterators...
-- 
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

>>>The basic idea of STL is that
>>>algorithms are written once and can then be applied to all data
>>>structures having a reasonable representation
>>>(e.g. can be traversed
>>>in some form or have some form of random access, depending on the
>>>algorithm).
>>
>>This might be true for some non-mutating algorithms.
> 
> 
> The rvalue reference proposal - library recommendations ( 
> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1771.html ) 
> addresses algorithms as well as the containers.  Check out the number of 
> mutating algorithms listed in there that do not require 
> CopyConstructible or CopyAssignable.  They include:
> 
> remove
> remove_if
> unique
> reverse
> rotate
> random_shuffle
> partition
> stable_partition
> sort
> stable_sort
> partial_sort
> nth_element
> inplace_merge
> the heap algorithms
> the permutation algorithms

Well I am glad that somebody else finaly verified and suggested some of 
concepts I was using for years.

Anyway, a the time being, this is just proposal. At the moment, there is 
no other way how to do things right than to avoid STL and use something 
else. Current STL algorithms do not allow storing other data than those 
with CopyConstructible or CopyAssignable.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <3ksapkFv949pU1@individual.net>, Mirek Fidler <cxl@volny.cz> 
wrote:

> > | Well, I am saying
> > | 
> > | Array<oftream> dumpfiles;
> > | 
> > | or
> > | 
> > | ArrayMap<String, ofstream> dumpfiles_map;
> > | 
> > | for more than 7 years now.
> > 
> > I know how to say
> > 
> >    MyArray<ofstream> dumpfiles(max_dump_index);
> > 
> > for more than 7 years.  That wasn't the point.
> 
> Well, maybe I got your point wrong, but please notice the absence of 
> "max_dump_index" in my post...

Below is working code on my desk.  I used ostringstream instead of 
ofstream, just for ease in creating a demo with output, no other reason:

#include <sstream>
#include <iostream>
#include <vector>
#include <string>
#include <algorithm>

struct pred
{
    bool operator()(const std::ostringstream& os)
    {
        return os.str()[0] == '2';
    }
};

int main()
{
    const int n = 5;
    std::vector<std::ostringstream> dumpfiles;
    std::string label[5] = {"one", "two", "three", "four", "five"};
    for (int i = 0; i < n; ++i)
    {
        std::ostringstream os;
        os << (i+1) << " : ";
        dumpfiles.push_back(std::move(os));
        dumpfiles.back() << label[i];
    }
    dumpfiles.erase(
       std::remove_if(dumpfiles.begin(), dumpfiles.end(), pred()),
       dumpfiles.end()
    );
    for (std::vector<std::ostringstream>::iterator i = dumpfiles.begin(),
            e = dumpfiles.end(); i < e; ++i)
    {
        std::cout << i->str() << '\n';
    }
}

The output is:

1 : one
3 : three
4 : four
5 : five

Notes:

1.  vector<ostringstream> !!!
2.  No max_dump_index.
3.  The ostringstream is manipulated both outside of, and inside of the 
container.  ostringstream state is transferred into the container.
4.  std::remove_if is used to manipulate the vector<ostringstream>.
5.  ostringstream is still neither CopyConstructible nor CopyAssignable.
6.  Syntax is nearly indistinguishable from today's syntax.  There is 
very little new to learn here.  Things just work.
7.  ostringstream is not special.  You can do the same thing with your 
movable but non-copyable types.  Just give them a move constructor and 
move assignment (typically less difficult than coding a copy constructor 
and copy assignment).

-Howard

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler <cxl@volny.cz> writes:

| > | > I can say
| > | > 
| > | >       ofstream dumpfiles[max_dump_index];
| > | > 
| > | > or
| > | > 
| > | >      new ofstream[max_dump_index];
| > | > 
| > | > yet, not
| > | > 
| > | >      vector<ofstream> dumpfiles;
| > | 
| > | Well, I am saying
| > | 
| > | Array<oftream> dumpfiles;
| > | 
| > | or
| > | 
| > | ArrayMap<String, ofstream> dumpfiles_map;
| > | 
| > | for more than 7 years now.
| > 
| > I know how to say
| > 
| >    MyArray<ofstream> dumpfiles(max_dump_index);
| > 
| > for more than 7 years.  That wasn't the point.
| 
| Well, maybe I got your point wrong,

Indeed, twice! :-)

| but please notice the absence of "max_dump_index" in my post...

It does not matter much to the point.

-- 
                                                       Gabriel Dos Reis 
                                           gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
>> The really major difference is that you are addressing something
>> entirely different than STL does! You are not at all addressing
>> algorithms which is the primary goal of STL.  You have some containers
>> as has STL but the containers for STL, albeit useful, are mostly
>> there as examples of sequences.
> 
> Dietmar, you was saying this before, but I think that this is not true.

I stick to what I said. In fact, I would even go a step further: the
real contribution of STL is entirely immaterial in C++! The heart of
STL are the concepts - even if they are, well, suboptimal (see my reply
to Scott's article for details). The next important step are the
algorithms: these are applicable to all sequences for which suitable
iterators can be created. ... and containers and iterators can be
provided however suitable: you can take the existing ones or the user
can create own sequences. Actually, a user can just create iterators
for existing containers never intended to work with STL and this
should work.

> STL is a library of containers and algorithms. Containers in STL are not
> "examples of sequences".

You mean, they are more than mere examples. Well, they are useful but
where they don't fit, a user can use his own. Actually, the is the
whole reason for STL's existence. Although it is not really spelled out
that explicitly, I think it is implicitly stated in the original
proposal from Alexander Stepanov and Meng Lee (you can see this text
from <http://www.hpl.hp.com/techreports/95/HPL-95-11.html>).

>> I'd bet that your containers could
>> also be used with STL algorithms.
> 
> Yes, as long as types stored satisfy general STL requirements, which
> often is not true.

The algorithms make pretty low requirements altough the requirements
are sometimes stronger than necessary. For example, there is no
inherent reason to exclude proxy sequences when moving objects. This
is one of the suboptimal areas of the current STL and there are
different approaches under discussion to remove unnecessary restrictions.
One of them is the r-value proposal and the property map/cursor stuff
is another, both addressing different restrictions.

> That is why I had to develop some algorithms anyway, 
> as they are guaranteed to work with types with different set of
> requirements.

The current concepts are flawed in several areas which is partly due
to lack of experience when they were introduced. I think the concepts
of STL should be reworked although the result will not look much
different than the current version, i.e. it will not work in terms of
indices but in terms of "cursors" which are quite similar to iterators
(actually, iterators are valid cursors but the dereference operation
is interpreted not to yield a value but rather a key). This is because
the algorithms shall be applicable to all sequences, including
sequences allowing O(1) insertions at arbitrary places which cannot
achieved while having O(1) index access, AFAIK.

Effectively, algorithms always have some inherent requirements on the
data structure, independent of how they are really implemented.
Although the current STL concepts come pretty close to a common set
of basic abstractions, the current concepts are sometimes too
restrictive. In other places, the specification of algorithms leaves
too much leeway for the implementer how to do things and thus has too
strong requirements. However, I think we are at a position where we
have more insight in the basic requirements and changing the concepts
to relax the restrictions or add more flexibility should move us in
the direction of have more universally applicable algorithms. In this
process it is probably a good idea to determine where your need to
have own algorithms was genuine and to check whether this need is
removed by the new concepts.

>> The basic idea of STL is that
>> algorithms are written once and can then be applied to all data
>> structures having a reasonable representation
>> (e.g. can be traversed
>> in some form or have some form of random access, depending on the
>> algorithm).
> 
> This might be true for some non-mutating algorithms.

It should be true for all algorithms. Due to several known restrictions
with the current STL concepts, it may not be true for the current STL.
However, the STL was the first truly generic library and when it was
specified and standardized it could not even be tested at all. Thus I
think it is more than like that some restrictions are simply there
which shouldn't. I'm still convinced that the basic idea of STL is the
correct direction for algorithms and that its goal can indeed be
achieved.
-- 
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

<wij@seed.net.tw> wrote in message 
news:1122556023.238692.88030@g47g2000cwa.googlegroups.com...
> Hendrik Schober wrote:
>> Er...because you do /not/ know how to program
>> in C++ unless you learned how to use the STL?
>
> 1. It is invalid to conclude that way beside untrue premise.
> 2. Must I use STL, or my program is less C++?
> 3. STL is good for many classes 'that fit its requirements'.
>   (many classes I am currently using simply do not fit)

Let me just add, that learning STL is a very good way of learning C++ 
generic programming. You, of course, can achieve with boost or other 
libraries the same or better result, but it wouldn't be easier. So if a 
person doesn't understand most part of STL then he probably missed a hole 
lot of C++. Just a thought.

- gene 


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Dietmar Kuehl <dietmar_kuehl@yahoo.com> writes:

| Gabriel Dos Reis wrote:
| 
| > "White Wolf" <wolof@freemail.hu> writes:
| > 
| > | In what way are indices more likely to stay valid than iterators?  I
| > | mean if I have index #10 tored away for a vector, and then I remove
| > | stuff and have only 5 elements, my index becomes rogue just the same way
| > | an iterator would. I just do not see (or believe) that it is any less
| > | likely with indices than iterators.
| > 
| > push_back is most likely to invalid previous iterators than shrinking
| > a vector.
| 
| Actually, it is easy to create a container similar to vector which
| does not suffer from invalidating iterators while mutating the
| contents: the iterators would simply store a pointer to the container
| and an index. In fact, you can even take 'std::vector' for this but
| use different than the normal iterators...

I know.  But that is an entirely different semantics guarantee than that
of the standard vector + iterator and I was responding to the claim above.

Allwoing vector<T>::iterator to be plain "T*" and allowing "T*" to be
mapped directly to comon hardware thingy directly weaken the
usuability of vector in terms of iterators.  For practical uses, I
maintain a (container, index) pair.

-- 
                                                       Gabriel Dos Reis 
                                           gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Howard Hinnant wrote:
> The rvalue reference proposal - library recommendations (
> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1771.html )
> addresses algorithms as well as the containers.  Check out the
> number of  mutating algorithms listed in there that do not require
> CopyConstructible or CopyAssignable.  They include:

Excuse me for responding without having first read the library
proposal that you linked to first, but...

> remove
> remove_if
> unique
> reverse
> rotate
> random_shuffle
> partition
> stable_partition
> sort
> stable_sort
> partial_sort
> nth_element
> inplace_merge
> the heap algorithms
> the permutation algorithms

....but how do you implement any of these... especially the shuffles and
sorts... without being able to move the elements from one location to
another?

The answer has to be std::swap, right? But the general implementation
of
swap requires T to be assignable.

If you aren't trying to prove some point, would you ever create a class
that can be swapped but not copied or assigned? Have you ever done this?


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Jeff Flinn wrote:

> Agreed. This is a quantum improvement in simplification/readability of user
> code! Awesome work Peter. Is this also part of TR1?

No, TR1 is pretty much final at this point. Nothing can go in, and it's
not clear what is the official way to extend it. I intend to propose
the extension for tr1::bind nonetheless. :-)


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> Below is working code on my desk.  I used ostringstream instead of 
> ofstream, just for ease in creating a demo with output, no other reason:
> 
> #include <sstream>
> #include <iostream>
> #include <vector>
> #include <string>
> #include <algorithm>
> 
> struct pred
> {
>     bool operator()(const std::ostringstream& os)
>     {
>         return os.str()[0] == '2';
>     }
> };
> 
> int main()
> {
>     const int n = 5;
>     std::vector<std::ostringstream> dumpfiles;
>     std::string label[5] = {"one", "two", "three", "four", "five"};
>     for (int i = 0; i < n; ++i)
>     {
>         std::ostringstream os;
>         os << (i+1) << " : ";
>         dumpfiles.push_back(std::move(os));
>         dumpfiles.back() << label[i];
>     }
>     dumpfiles.erase(
>        std::remove_if(dumpfiles.begin(), dumpfiles.end(), pred()),
>        dumpfiles.end()
>     );
>     for (std::vector<std::ostringstream>::iterator i = dumpfiles.begin(),
>             e = dumpfiles.end(); i < e; ++i)
>     {
>         std::cout << i->str() << '\n';
>     }
> }
> 
> The output is:
> 
> 1 : one
> 3 : three
> 4 : four
> 5 : five
> 
> Notes:
> 
> 1.  vector<ostringstream> !!!
> 2.  No max_dump_index.
> 3.  The ostringstream is manipulated both outside of, and inside of the 
> container.  ostringstream state is transferred into the container.

Well, "transfer" is the right term here, I agree :)

> 4.  std::remove_if is used to manipulate the vector<ostringstream>.
> 5.  ostringstream is still neither CopyConstructible nor CopyAssignable.
> 6.  Syntax is nearly indistinguishable from today's syntax.  There is 
> very little new to learn here.  Things just work.
> 7.  ostringstream is not special.  You can do the same thing with your 
> movable but non-copyable types.  Just give them a move constructor and 
> move assignment (typically less difficult than coding a copy constructor 
> and copy assignment).

You still require move constructor to be part of type stored. Far too 
much for me.

Code on my desk certainly fails in point 6. but allows things like

Array<Ctrl> ctrl; // Ctrl is base class for widget hierarchy
double numbers[] = { 1, 3.3, 4, 7.7, 1, 0.5 };
for(int i = 0; i < 6; i++)
    if(numbers[i] == (int)numbers[i])
        ctrl.Create<EditInt>() <<= numbers[i];
    else
        ctrl.Create<EditDouble>() << numbers[i];
Sort(ctrl, StdDataLess());
TopWindow dlg;
for(int i = 0; i < ctrl.GetCount(); i++)
    dlg.Add(ctrl[i].LeftPos(0, 200).TopPos(i * 20, 20);
dlg.SetRect(0, 0, 200, 200);
dlg.Run();

.... displays dialog with int and double input fields, sorted by numbers 
contained in them.

All that is required for types stored in Array<Ctrl> is that they are of 
type Ctrl or derived from it and have default constructor (default 
constructor requirement holds true only for Create method, can be avoided).

Of course, I could store ostringstream into Array as easily without 
difficulity and without std::move. E.g. (direct equivalent):

Array<ostringstream> x;
for(int i = 0; i < n; i++) {
    x.Add() << (i + 1) << " : ";
    x.Top() << label[i];
}

Does not require ostringstream to be movable (in your terminology). It 
just has to have default constructor, something easily satisfied by 
virtually any class. But even without it, Array is still applicable.

And note that it is significantly shorter code as well...

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Dietmar Kuehl <dietmar_kuehl@yahoo.com> writes:

[...]

| > STL is a library of containers and algorithms. Containers in STL are not
| > "examples of sequences".
| 
| You mean, they are more than mere examples. Well, they are useful but
| where they don't fit, a user can use his own.

One can argue about the value of having them standard if the library
and language constructs can not make them composable for the basic things
and people have tp continue to revinvent the wheel after so much of
excitement!  Howard reports a bridge in form of std::move.  That is 
encouraging news.  I'm looking forward to seeing better integration.
And yes, it is possible to reduce the neds to wheel reinvention.

I understand the view of "where they don't fit, a user can use his
own"; but I fear that it is being carried way to far, with the
tendency of "expert only" language and library that contains only
"cool stuff" and leaves the basic, frequent things to the user.

-- 
                                                       Gabriel Dos Reis 
                                           gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

>> for_each(clients.begin(), clients.end(), CancelAllSignatures());
>> I think it's much better than a loop. It's obvious we will cancel all
>> signatures from all clients.

>Is it? What CancelAllSignatures returns? Is it a functor?

Yes, its a functor.

>> But it's good because the code reader
>> doesn't need to know the implementation detail.

>Impementation detail of what? for_each? Come on.

No, implementation details of the functor CancelAllSignatures. It's
clear all signatures will be cancelled, but the implementation is
"hidden". IMO it makes the code clear.

Rodrigo Strauss


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Dietmar Kuehl wrote:
> Gabriel Dos Reis wrote:
> 
> 
>>"White Wolf" <wolof@freemail.hu> writes:
>>
>>| In what way are indices more likely to stay valid than iterators?  I
>>| mean if I have index #10 tored away for a vector, and then I remove
>>| stuff and have only 5 elements, my index becomes rogue just the same way
>>| an iterator would. I just do not see (or believe) that it is any less
>>| likely with indices than iterators.
>>
>>push_back is most likely to invalid previous iterators than shrinking
>>a vector.
> 
> 
> Actually, it is easy to create a container similar to vector which
> does not suffer from invalidating iterators while mutating the
> contents: the iterators would simply store a pointer to the container
> and an index. In fact, you can even take 'std::vector' for this but
> use different than the normal iterators...

Sure. But it solves only invalidating of iterator, not out-of range problem.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> I stick to what I said. In fact, I would even go a step further: the
> real contribution of STL is entirely immaterial in C++!

Hehe, I think we can agree on that :)

> to lack of experience when they were introduced. I think the concepts
> of STL should be reworked although the result will not look much
> different than the current version, i.e. it will not work in terms of
> indices but in terms of "cursors" which are quite similar to iterators
> (actually, iterators are valid cursors but the dereference operation
> is interpreted not to yield a value but rather a key).

Well, actually, at the moment I have still based my algorithms mostly on 
iterator concept (just with different set of requirements). But I must 
say that right at the moment, I am starting to think it is not that good 
idea - I have encountered a several situations where this concept leads 
to inherently ineffective code or iterators do not provide sufficient 
interface to create required algorithm at all.

That is why I have started thinking about swithing to some different 
approach, perhaps based on some general container interface. In the end, 
you can always transform begin - end pair to the container 
representation....

> This is because
> the algorithms shall be applicable to all sequences, including
> sequences allowing O(1) insertions at arbitrary places which cannot
> achieved while having O(1) index access, AFAIK.

Ah, famous list argument :)

Well, first of all, number of really usefull algorithms applicable on 
list is quite limited. (But this opinion is likely to be influenced by 
my opposition to implement trivial loops by algorithms and functors).

Anyway, more importantly, I believe that O(1) insertions advantage is a 
myth. Before you can apply such insertion, you have to find place where 
to insert and list iteration is much slower than vector iteration.

I am afraid that number of real world problems where list has any 
advantage is pretty close to zero. I have not found any real use for 
list container since I have started using C++ (for 12 years). (BTW, this 
does not apply at list data structure, just as container it is useless).

> the direction of have more universally applicable algorithms. In this
> process it is probably a good idea to determine where your need to
> have own algorithms was genuine and to check whether this need is
> removed by the new concepts.

OK. Most important differences are:

a) my algorithms allow T to have implicit move copy (a = b; - b can now 
be "moved", well, in my terminology, "picked")

b) sort requires just iter_swap and compare predicate.

And well, I do provide syntax sugar so that algorithms can be applied on 
entire container, but that can be done for STL easily (but makes me 
wonder why it was not? what is so special on typing "foo.begin(), 
foo.end()" all over again?)

> which shouldn't. I'm still convinced that the basic idea of STL is the
> correct direction for algorithms and that its goal can indeed be
> achieved.

Well, depends on what you think that basic idea of STL is...

If it is idea of containers that do not require elements stored to be 
derived from some "Object", I agree.

If it is idea that algorithms should be separated from containers 
universal, I agree.

With iterators as universal sequence representation, I am not so sure, 
but I can agree to some degree, at least before I will know some better 
solution.

With STL set of requirements and containers I am very unsatisfied.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Russell Hind wrote:
> Peter Dimov wrote:
> >
> > The current CVS version of boost::bind (soon to be released) supports
> >
> > bind(&Excipient_c::GetDescription, _1) == Name
> >
> > and
> >
> > bind(&Excipient_c::GetDescription, _1) <
> > bind(&Excipient_c::GetDescription, _2)
>
> Thanks, I didn't realise this.  That will help a lot.
>
> Just out of interest, why isn't this sort of thing mentioned in the
> latest news for 1.33.0
> (http://cvs.sourceforge.net/viewcvs.py/*checkout*/boost/boost/index.htm?rev=1.230)?

It should be now. :-)


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Hendrik Schober wrote:
> wij@seed.net.tw <wij@seed.net.tw> wrote:
> 
>>[...]
>>Let's say, If I already know how to program in C++, why
>>bother spending longer time to learn STL [...]
> 
> 
>   Er...because you do /not/ know how to program
>   in C++ unless you learned how to use the STL?
> 

That is just plain wrong. I would guess that in fact most C++ 
programmers do not use the STL. Think about the 100s of thousands of C++ 
programmers who use MFC, ATL, VCL, Qt, CLI...
All of these libraries provide alternatives to the STL and most were 
around long before the STL became standard. Just because someone picks a 
third-party library and decides to make it part of the C++ standard does 
not mean people have to use it.

Andrew.

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler <cxl@volny.cz> writes:

| Dietmar Kuehl wrote:
| > Gabriel Dos Reis wrote:
| > 
| > 
| >>"White Wolf" <wolof@freemail.hu> writes:
| >>
| >>| In what way are indices more likely to stay valid than iterators?  I
| >>| mean if I have index #10 tored away for a vector, and then I remove
| >>| stuff and have only 5 elements, my index becomes rogue just the same way
| >>| an iterator would. I just do not see (or believe) that it is any less
| >>| likely with indices than iterators.
| >>
| >>push_back is most likely to invalid previous iterators than shrinking
| >>a vector.
| > 
| > 
| > Actually, it is easy to create a container similar to vector which
| > does not suffer from invalidating iterators while mutating the
| > contents: the iterators would simply store a pointer to the container
| > and an index. In fact, you can even take 'std::vector' for this but
| > use different than the normal iterators...
| 
| Sure. But it solves only invalidating of iterator, not out-of range problem.

Isn't that one already solved?

-- 
                                                       Gabriel Dos Reis 
                                           gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> Anyway, more importantly, I believe that O(1) insertions advantage is a
> myth. Before you can apply such insertion, you have to find place where
> to insert and list iteration is much slower than vector iteration.
>
> I am afraid that number of real world problems where list has any
> advantage is pretty close to zero. I have not found any real use for
> list container since I have started using C++ (for 12 years). (BTW, this
> does not apply at list data structure, just as container it is useless).

I am really puzzled.  What can be the more natural representation of
the list data structure than the list container?

I use std::list a lot, for different purposes.  Very often it is just a
container of pointers to maintain some objects' lifetime.  Other
containers can do too, but I find list the easiest and the fastest.

Best regards,

Yongwei


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> Well I am glad that somebody else finaly verified and suggested some of
> concepts I was using for years.
>
> Anyway, a the time being, this is just proposal. At the moment, there is
> no other way how to do things right than to avoid STL and use something
> else. Current STL algorithms do not allow storing other data than those
> with CopyConstructible or CopyAssignable.

What makes you uncomfortable with Container<Object*> or
Container<shared_ptr<Object> >?  Please name some places where your NTL
containers have an advantage over them.

Best regards,

Yongwei


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
>> I know how to say
>>
>>    MyArray<ofstream> dumpfiles(max_dump_index);
>>
>> for more than 7 years.  That wasn't the point.
>
> Well, maybe I got your point wrong, but please notice the absence of
> "max_dump_index" in my post...

I think you got Gabriel's point indeed wrong: I think, he is saying
that he nows how to create his own container dealing with objects of
type 'ofstream'. It is, however, an embarrassment that the standard
containers don't do the same. The presence or absence of the initial
container size is quite irrelevant in this context.

However, why focus on containers anyway? Scott specifically asked
about the *algorithms*! It is much more important to makes these
universally usable anyway: everybody can write his own sequences.
I, at least, are quite used to creating my own sequences, most often
in the form of input iterators giving some view of containers, and
to use them with algorithms. In this context I find too often that
the current STL concepts are too restrictive.
-- 
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> Gabriel Dos Reis wrote:
>> Mirek Fidler <cxl@volny.cz> writes:
>>
>> | Containers in STL are not "examples of sequences".
>>
>> What are they?
>>
> Ehm, pardon my english, perhaps I got bad context.

Well, maybe Gabriel understands the containers as examples as I do
and his question is genuinely "what are they, if not [only] examples?"

> What I wanted to say is that I think that Dietmar thinks that STL
> containers are unimportant

.... from the perspective of STL. Users may have and typically do have
a different view on these containers as is clearly demonstrated by
repeated requests to add certain containers. Most of these are easy
to write and it is easier to just create them than to complain about
their absence. After all, all a container has to do is to cope with
the stuff immediately related to containment (element maintenance and
access). The algorithms operating on containers are readily available
for user defined containers.

> and are just examples for you how to build your own.

No, this is not at all what I think nor, hopefully, said! Of course,
you can create your containers the way the STL containers are build.
However, you can use drastically different approaches, too. The
containers are examples of different sequences, all applicable to
STL algorithms. The different selections, e.g. array base and node
based, demonstrate that the algorithms work with all of them. I guess,
even 'std::bitset' and 'std::vector<bool>' where included in the
believe that it demonstrates that STL algorithms can operate on them.
Unfortunately, it turned out that the STL concepts are flawed and e.g.
do not allow proxy sequences. This is, of course, something which can
be fixed by changing the basic concepts. There was a proposal relaxing
the current requirements but I think that it is understood that the
separation of traversal and access yields a better fix.

As I said in another article: STL is the right way to go. It needs,
however, some tweaking with its concepts to become universally
applicable (plus some usability stuff...).
-- 
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

E. Rosten wrote:
> If you append stuff to a std::vector, iterators become invalid, indices do
> not.

Actually, as stated this statement is wrong: not all iterators become
invalidated when appending stuff to a 'std::vector' (and, no, I'm not
nitpicking on the situation where the capacity is sufficient): it is
easy to create an iterator class which stores e.g. a pointer to the
vector and an index. Appending to the vector will not invalidate this
iterator at all. It is only a question of whether you are willing to
go through an indirection or not. Actually, I can see use-cases for
both and it may be reasonable to provide an corresponding iterator
class, e.g. in the form of a template taking the container type as
template argument.

>> 3.) Unless the container itself is being changed inside the loop, the
>> end() iterator is better be saved away into a const variable before the
>> loop.
>
> This is no longer true:

How do you know how I have implemented my container? It may not be
true for one specific container, however, it will almost certainly
be true for some containers. ... and in the realm of generic
programming irrelevant details like the currently used container type
is subject to change. Thus, it is important that assertions are true
for all possible entities, not just for the current one.
-- 
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
>> I stick to what I said. In fact, I would even go a step further: the
>> real contribution of STL is entirely immaterial in C++!
>
> Hehe, I think we can agree on that :)

Note the difference between "irrelevant" and "immaterial". What I
mean is that the real contribution of STL, i.e. the concepts, are
not really expressed in [current] C++ source code at all.

> Well, actually, at the moment I have still based my algorithms mostly on
> iterator concept (just with different set of requirements). But I must
> say that right at the moment, I am starting to think it is not that good
> idea

I agree that the iterator concept is not chosen well. It should have
split into two separate concepts: property maps and cursors. However,
it appears you are thinking of something different:

> That is why I have started thinking about swithing to some different
> approach, perhaps based on some general container interface. In the end,
> you can always transform begin - end pair to the container
> representation....

A containers interface is the wrong level of abstraction, at least
for my uses. Many of the sequences I process are not directly container
and e.g. do not allow insertion or removal of elements. In fact, most
of these sequences are actually views on other sequences which filter
or extend the underlying sequence. A range concept could be feasible
for these sequence but conceptually ranges are not different than pairs
of iterators. Whether to use ranges or iterator pairs is mostly a
usability issue and one that has arguments in favor of both positions
(...and thus the conclusion is to provide both which is not yet done
in the standard library).

> Anyway, more importantly, I believe that O(1) insertions advantage is a
> myth. Before you can apply such insertion, you have to find place where
> to insert and list iteration is much slower than vector iteration.

It is definitely not a myth in the area of graph algorithms where it
is quite conventional to insert and remove e.g. auxiliary edges on
the fly. The position to insert or remove at is naturally obtained
from the current position which in turn is e.g. immediately, without
traversing the list at all, obtained from an entering edge.

> I am afraid that number of real world problems where list has any
> advantage is pretty close to zero. I have not found any real use for
> list container since I have started using C++ (for 12 years).

Maybe this is due to your view? My view is different and personally
I have put lists to good use in the past (although these had not
necessarily been 'std::list' but some form of circular list).

>> the direction of have more universally applicable algorithms. In this
>> process it is probably a good idea to determine where your need to
>> have own algorithms was genuine and to check whether this need is
>> removed by the new concepts.
>
> OK. Most important differences are:
>
> a) my algorithms allow T to have implicit move copy (a = b; - b can now
> be "moved", well, in my terminology, "picked")
>
> b) sort requires just iter_swap and compare predicate.

To meet these requirements I can only envision the following
alternatives for your containers:

- They are always based on node based structures (e.g. each element is
   actually represented by a pointer to the element in the container).
   For the allocation of nodes exists different alternatives, in
   particular, it is not necessary to allocate each node individually.
   However, this defeats certain deliberate goals like colocation of
   objects, e.g. to fit into only few cache lines.
- You can only operate on PODs with the obvious and known restrictions.
- You use non-portable approaches like memcopy()ing non-POD objects.

All of these restrictions are IMO much more severe than having explicit
move or even copy semantic. Maybe I don't see another alternative or
we will have to agree that our measures for restrictions are vastly
different.

> Well, depends on what you think that basic idea of STL is...
>
> If it is idea of containers that do not require elements stored to be
> derived from some "Object", I agree.

I do not really care at all for containers (as mentioned before) and
I still think that containers are entities dealt with by the primary
STL subject (algorithms). Of course, I prefer containers where there
are only reasonable restrictions on the element types and having to
derive from whatever particular base class is not reasonable
restriction.

> If it is idea that algorithms should be separated from containers
> universal, I agree.

I think this is the heart of STL: separation from algorithms and
data structures. The data structure should only be concerned with
their internal representation and should not be concerned with
algorithms operating on them. To make this viable, the data
structures need to provide access through some level of indirection
which implements certain concepts.

> With iterators as universal sequence representation, I am not so sure,
> but I can agree to some degree, at least before I will know some better
> solution.

As mentioned above, I think the major flaw of iterators is that
they combine traversal and access into one abstraction. This should
be separated. There are also some abstractions missing or a lumped
unnecessarily into just one concept: for certain operations it would
be desirable to have immediate sequence mutating operations, i.e.
ability to add or remove elements. The container abstraction could
be used for this but I don't think this is a good fit at all:
containers do much to much for this.

> With STL set of requirements and containers I am very unsatisfied.

Except for the missing separation I consider the concepts OK,
especially if the move stuff becomes supported by the language. For
my needs the containers are mostly suitable although I sometimes
prefer a hash over a map. Also, I create custom containers where
necessary.
-- 
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Allan W wrote:
> Excuse me for responding without having first read the library
> proposal that you linked to first, but...

You should have read the proposal and its precondition (the rvalue
and move semantic stuff) first:

> ...but how do you implement any of these... especially the shuffles and
> sorts... without being able to move the elements from one location to
> another?

You don't implement them with the given constraints. You implement them
in terms of move semantics, where applicable, and the current semantics
where not. That is, the restriction of being copy constructible and
assignable is relaxed to rely on either movable objects or the
restrictions which are in place now.

> The answer has to be std::swap, right?

No. The answer is move semantics and a move constructor.
-- 
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Andrew Ward wrote:
> Hendrik Schober wrote:
>>   Er...because you do /not/ know how to program
>>   in C++ unless you learned how to use the STL?
>>
>
> That is just plain wrong.

Actually, I disagree: if you don't know the STL way, whether you
need it or not, you don't know how to use all the C++ potential.
Essentially, STL is an instance of a particular programming
paradigm (Generic Programming) which is essentially only applied in
the context of STL. Maybe Hendrik's statement is a little bit too
strong, i.e. you can do effective work in C++ without knowing STL,
but you definitely do not know all of C++ worth knowing.

> I would guess that in fact most C++ programmers do not use the STL.

Ignorance has never been a good indicator for knowledge, especially
if it was only accidental and not deliberate.

> Think about the 100s of thousands of C++ programmers who use MFC,
> ATL, VCL, Qt, CLI...

It is an issue beyond what your are using. The basic ideas of most
of these libraries are quite similar and rather different than those
of STL, although some of these libraries have embraced at least some
portions of STL. Essentially, I would claim that the foundation of
all the libraries you mention and of most the libraries you didn't
is object orientation. STL's foundation is an entirely different
paradigm although it assumes some aspects of object orientation (e.g.
encapsulation). But frowns upon others (e.g. dynamic polymorphism)
while still being even applicable in the face of these.

> All of these libraries provide alternatives to the STL

Actually, they don't. They may provide containers as STL does but
STL is not about containers. STL is about algorithms which are
entirely independent of containers. They operate on sequences and
containers happen to be examples of these sequences. I don't think
that any of the libraries you mention and most of the libraries you
didn't provides an extensive set of algorithms. Hence, they are no
alternative to STL.

> and most were
> around long before the STL became standard. Just because someone picks a
> third-party library and decides to make it part of the C++ standard does
> not mean people have to use it.

Actually, this "someone" was a committee with all big players in
the C++ area being represented and there was obviously a majority
to pick this specific library and not others. Of course, this does
not mean that you have to use it but it is, IMO, an indication that
this library is at least worth looking at when working with C++. Be
careful though: doing so might change your view on what STL is.
-- 
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> Dietmar Kuehl wrote:
>> Actually, it is easy to create a container similar to vector which
>> does not suffer from invalidating iterators while mutating the
>> contents: the iterators would simply store a pointer to the container
>> and an index. In fact, you can even take 'std::vector' for this but
>> use different than the normal iterators...
>
> Sure. But it solves only invalidating of iterator, not out-of range
> problem.

Are you confident that it is impossible to solve the out-of-range
problem the same way, e.g. by including appropriate checks with the
corresponding operations? There are many other things you can put
into such iterators, e.g. you can create an iterator which has a
designated singular value you can test against.

The only real problem with iterators is processing multiple sequences
simultaneously: you have to keep a separate iterator for each
sequence. However, this problem is also addressed by the separation
of traversal and access: the different sequences would simply be
viewed as one sequence and thus only one cursor would be involved.
To actually access the different attributes, different property maps
are used. In this case, the cursor could e.g. be essentially an
index (of course, since cursors need dereferencing to provide the
key, the index would be wrapped with a small class). There are
other alternatives for the representation of the cursor, too (e.g.
a set of simultaneously kept iterators; the difference is that to the
algorithms it is just one object; in this case the dereference
operation would simply return the original cursor and the property
maps would know how to extract their specific iterator).
-- 
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> Array<ostringstream> x;
> for(int i = 0; i < n; i++) {
>     x.Add() << (i + 1) << " : ";
>     x.Top() << label[i];
> }
>
> Does not require ostringstream to be movable (in your terminology). It
> just has to have default constructor, something easily satisfied by
> virtually any class. But even without it, Array is still applicable.
>
> And note that it is significantly shorter code as well...

But isn't your Array<T> type just a dynamic array of pointers to T
(even though the pointer is hidden)? I think Howard's feat is more
impressive (and ultimately more useful/applicable) because it works
with values, not pointers to values. You seem to have missed the fact
that his example wrote to a stream, moved the stream into the vector,
and then wrote to the (moved) stream again.

Bob


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

I got the following from Bjorn Reese:

I am sorry for replying directly to you rather than to the newsgroup.
I am having difficulties posting on comp.lang.c++.moderated (and I
haven't got the faintest clue why).

The message below is not private; you are allowed to quote the message,
in part or in full, on the newsgroup. If you do, however, I would
appreciate it if you would scramble my email address (to avoid spam).


Scott Meyers wrote:

> From my perspective, there are two questions here.  One is, as I said,
> whether it is practical to replace many manual loops with algorithm
> calls.

One of the "selling-points" of algorithm calls is that each call has a
descriptive name that indicates the purpose of the call. So, if you use
find_if(), then others can immediately see that you are looking for the
first element that fits a given criterion. For example:

     nail = find_if(haystack.begin(), haystack.end(), isNail);

Now, for this to be a real selling-point, it has to claim either that
algorithm calls are comprehended more quickly than for-loops, or that
programmers have more difficulty recognizing the purpose of an
equivalent for-loop like:

     for (nail = haystack.begin(); nail != haystack.end(), ++nail)
       if (isNail(*nail))
         break;

I recently conducted a study that, amongst others, investigates this
claim. The study has not been published yet, as I am still analyzing
the results.

One of the results of the study showed that programmers read
stereotypical loops with ease (e.g. counting, finding, replacing). It
is worth mentioning here that STL only provides algorithm calls for
stereotypical loops. By ease, I mean that the programmers only read
such loops once, without having to mentally execute parts of the loop.

Another result of the study showed that programmers had no problems
recognizing the purpose of for-loops in a categorization task.

In summary, the empirical data suggest that there is no noticable
advantage of algorithm calls over for-loops with regards to
comprehention speed nor with regards to recognizing the purpose
of a stereotypical loop.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Howard Hinnant wrote:

[snip code example]

> 1.  vector<ostringstream> !!!
> 2.  No max_dump_index.
> 3.  The ostringstream is manipulated both outside of, and inside of the
> container.  ostringstream state is transferred into the container.
> 4.  std::remove_if is used to manipulate the vector<ostringstream>.
> 5.  ostringstream is still neither CopyConstructible nor CopyAssignable.
> 6.  Syntax is nearly indistinguishable from today's syntax.  There is
> very little new to learn here.  Things just work.
> 7.  ostringstream is not special.  You can do the same thing with your
> movable but non-copyable types.  Just give them a move constructor and
> move assignment (typically less difficult than coding a copy constructor
> and copy assignment).

Pretty darn cool!

Question: what happens if you use "os" after it's moved?

    for (int i = 0; i < n; ++i)
    {
        std::ostringstream os;
        os << (i+1) << " : ";
        dumpfiles.push_back(std::move(os));

        os << "hello";
    }

Bob


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Harald Luessen was seen penning the following ode to ... whatever:
> What everybody in the thread did first was to show the corresponding
> loop, a three-liner, and nobody asked about its meaning. Everybody
> understood immediately. Imagine the not so skilled collegues who
> may have to understand and change your code.

While this is something that needs to be kept in mind, you should also
consider where you draw the line - do you "dumb down" the C++ experts
in your team to the level of the weakest/least experienced member, do
you require a minimum level of knowledge instead or do you "force" the
less experienced members of the team to catch up with the gurus?

For me this is a bit of a sore point - I've been critised in the past
for using STL/boost/whatever as some other team members didn't
understand the code and didn't want to learn about them either (as
they didn't see a benefit in doing so) but I am a strong believer in
making use of as many tools as are sensible - after all, why should I
go an write something that's already in the standard C++ library or
boost if I can just reuse it?

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> | Sure. But it solves only invalidating of iterator, not out-of range problem.
>
> Isn't that one already solved?

Well, I am addressing issue when some elements from container are
removed while performing loop, compare

for(std::vector<foo>::iterator it = bar.begin(); it != bar.end(); ++it)
     if(condition(*it))
        bar.erase(it);

for(int i = 0; i < bar.GetCount(); i++)
     if(condition(bar[i]))
        bar.Remove(i);

(not that above code would be useful, but I hope it helps to illustrate
the problem).

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> The answer has to be std::swap, right? But the general implementation
> of
> swap requires T to be assignable.

Well, that is the point of proposal. Swap will not require T to be
assignable. Move will be all that is needed.

> If you aren't trying to prove some point, would you ever create a class
> that can be swapped but not copied or assigned? Have you ever done this?

Well, I am using library that exploits principles somewhat similar to
what this proposal provides and yes, I have a lot of classes that can be
swapped but not copied or assigned. All of my containers are of this
kind. Makes life much easier.

Moreover, std::sort can be implemented, without any performance issues,
with even lower requirements - all you need is in fact iter_swap and
comparing predicate.

This way I am able to provide containers capable of storing of
polymorphic elements to be sorted using standard algorithm.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Andrew Ward <andy.ward@ihug.co.nz> wrote:
> Hendrik Schober wrote:
> > wij@seed.net.tw <wij@seed.net.tw> wrote:
> >
> >>[...]
> >>Let's say, If I already know how to program in C++, why
> >>bother spending longer time to learn STL [...]
> >
> >
> >   Er...because you do /not/ know how to program
> >   in C++ unless you learned how to use the STL?
> >
>
> That is just plain wrong. I would guess that in fact most C++
> programmers do not use the STL. Think about the 100s of thousands of C++
> programmers who use MFC, ATL, VCL, Qt, CLI...
> All of these libraries provide alternatives to the STL and most were
> around long before the STL became standard. Just because someone picks a
> third-party library and decides to make it part of the C++ standard does
> not mean people have to use it.

  C++ is more than just the core language. The
  std library is an integral part of it and
  the two are woven into each other at many
  places.
  Saying "I know C++, but I don't know the STL"
  for me sounds the same as saying "I know C++,
  but I don't know about OO".

> Andrew.

  Schobi

-- 
SpamTrap@gmx.de is never read
I'm Schobi at suespammers dot org

"Coming back to where you started is not the same as never leaving"
 Terry Pratchett



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Dietmar Kuehl <dietmar_kuehl@yahoo.com> writes:

| Mirek Fidler wrote:
| >> I know how to say
| >>
| >>    MyArray<ofstream> dumpfiles(max_dump_index);
| >>
| >> for more than 7 years.  That wasn't the point.
| >
| > Well, maybe I got your point wrong, but please notice the absence of
| > "max_dump_index" in my post...
| 
| I think you got Gabriel's point indeed wrong: I think, he is saying
| that he nows how to create his own container dealing with objects of
| type 'ofstream'. It is, however, an embarrassment that the standard
| containers don't do the same. The presence or absence of the initial
| container size is quite irrelevant in this context.

Absolutely.

| However, why focus on containers anyway? Scott specifically asked
| about the *algorithms*!

You're quite right.  However, I'd observe that we're divergeing 
talking about containers because algorithms need to work on
"something" and that "something" usually is a container :-)

-- 
                                                       Gabriel Dos Reis 
                                           gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Allan W" <allan_w@my-dejanews.com> writes:

| The answer has to be std::swap, right? But the general implementation
| of
| swap requires T to be assignable.
| 
| If you aren't trying to prove some point, would you ever create a class
| that can be swapped but not copied or assigned? Have you ever done this?

I think an observation made earlier (not in this thread, but in the
move semantics) is that swap does not need assignable.  If your
objects are movable, you can swap too :-)


-- 
                                                       Gabriel Dos Reis 
                                           gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler <cxl@volny.cz> writes:

> That is why I have started thinking about swithing to some different 
> approach, perhaps based on some general container interface. In the end, 
> you can always transform begin - end pair to the container 
> representation....

In my current work I am working with a "Sequence" concept (yes, I know
the standard has one but it's pretty weak and I needed a name) that
bundles a property map and two cursors.  That seems to be simplifying
the interface and improving composability.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Andrew Ward wrote:
> Hendrik Schober wrote:
> > wij@seed.net.tw <wij@seed.net.tw> wrote:
> >
> >>[...]
> >>Let's say, If I already know how to program in C++, why
> >>bother spending longer time to learn STL [...]
> >
> >
> >   Er...because you do /not/ know how to program
> >   in C++ unless you learned how to use the STL?
> >
>
> That is just plain wrong. I would guess that in fact most C++
> programmers do not use the STL. Think about the 100s of thousands of C++
> programmers who use MFC, ATL, VCL, Qt, CLI...
> All of these libraries provide alternatives to the STL and most were
> around long before the STL became standard. Just because someone picks a
> third-party library and decides to make it part of the C++ standard does
> not mean people have to use it.

It's pretty close to correct.

You don't know how to use standard C++ if you don't know how to use the
STL. Period.

This doesn't mean that you have to _use_ the STL - just that you have
to _know_ how to use it well.

Hating the STL is fine (in a free speech sort of way) if it's based on
knowledge and understanding; most people, however, just try to hide
their own ignorance behind "STL sucks" slogans.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> What makes you uncomfortable with Container<Object*> or

Requires manual management of Object's life-time.

> Container<shared_ptr<Object> >?

A little bit better, but makes object life-time fuzzy.

> Please name some places where your NTL
> containers have an advantage over them.

See example posted in another post.

Anyway, if nothing else, Array<Object> is in any case much faster than 
vector< shared_ptr<Object> >.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Wu Yongwei wrote:
> Mirek Fidler wrote:
> 
>>Anyway, more importantly, I believe that O(1) insertions advantage is a
>>myth. Before you can apply such insertion, you have to find place where
>>to insert and list iteration is much slower than vector iteration.
>>
>>I am afraid that number of real world problems where list has any
>>advantage is pretty close to zero. I have not found any real use for
>>list container since I have started using C++ (for 12 years). (BTW, this
>>does not apply at list data structure, just as container it is useless).
> 
> 
> I am really puzzled.  What can be the more natural representation of
> the list data structure than the list container?

Well, it is just virtually each time I need linked list kind of thing, I 
always need it in non-container form.

E.g. I need to group objects regardless their actuall placement (they 
can by on heap, on stack, in container, whatever). Then double-linked 
list is very natural thing to use, but obviously deque will not do it... 
(because those objects are already owned).

> I use std::list a lot, for different purposes.  Very often it is just a
> container of pointers to maintain some objects' lifetime.  Other
> containers can do too, but I find list the easiest and the fastest.

Well, "fastest" depends. Have ever done actual measurements?

vector-like containers (esp. NTL Vector) always outperfrom list in 
operations like push_back or iterations.

Of course, insert and erase at arbitrary position is a problem for 
vector, but then again I think that my above statement about finding 
position in list is generally true. I mean, while list insert/erase 
itself is O(1), "where to insert" problem is likely to make total 
performance worse.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

>>Array<ostringstream> x;
>>for(int i = 0; i < n; i++) {
>>    x.Add() << (i + 1) << " : ";
>>    x.Top() << label[i];
>>}
>>
>>Does not require ostringstream to be movable (in your terminology). It
>>just has to have default constructor, something easily satisfied by
>>virtually any class. But even without it, Array is still applicable.
>>
>>And note that it is significantly shorter code as well...
> 
> 
> But isn't your Array<T> type just a dynamic array of pointers to T
> (even though the pointer is hidden)? I think Howard's feat is more
> impressive (and ultimately more useful/applicable) because it works
> with values, not pointers to values.

Array works with values. Otherwise you could apply your logic to all 
node based containers.

> You seem to have missed the fact
> that his example wrote to a stream, moved the stream into the vector,
> and then wrote to the (moved) stream again.

As for Howard's approach, that is good, important as well (and actually 
I have equivalent tools available), I would just never used it for 
streams...

The main problem with Howard's approach is that it does work only with 
elements with specific interface and does not solve polymorphic issues.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Wu Yongwei" <wuyongwei@gmail.com> writes:

| Mirek Fidler wrote:
| > Anyway, more importantly, I believe that O(1) insertions advantage is a
| > myth. Before you can apply such insertion, you have to find place where
| > to insert and list iteration is much slower than vector iteration.
| >
| > I am afraid that number of real world problems where list has any
| > advantage is pretty close to zero. I have not found any real use for
| > list container since I have started using C++ (for 12 years). (BTW, this
| > does not apply at list data structure, just as container it is useless).
| 
| I am really puzzled.  What can be the more natural representation of
| the list data structure than the list container?

One might argue whether one is considering "list" as synonymous of
"finite sequence" or "lits" as "linked list" :-)

| I use std::list a lot, for different purposes.

That is very interesting.  
Very recently, I was asked the question "do you know of a genuine use
of (linked) list that is simple to explain?"  Since the question was
asked by someone who obvisouly is not what is called a beginner, I
paused a bit before risking my first tentative (which turned out to be
"wrong" anyway).  Then, we spent a couple of days before reaching some
kind of agreements on something.  The key observation that killed most
of the examples was that each time one believes that a list is
needed/suitable, actually one was thinking of the abstract notion of
"sequence" or "list", not the "linked list" data structure.  Usually
the items are part of other bigger data structures and are chained
together by conceptual links and similar things. And a better and simpler
representation always seemed to exist.

So, this is to say that I'm highly interested in your frequent uses of
std::list both for my education and curiosity -- the key part i
"simple to explain".

| Very often it is just a
| container of pointers to maintain some objects' lifetime.  Other
| containers can do too, but I find list the easiest and the fastest.

Indeed, I'm wondering why std::vector would not have been a better
representation in terms of simplicity and efficiency.

-- 
                                                       Gabriel Dos Reis 
                                           gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Timo Geusch <tnews@unixconsult.co.uk> wrote:
> Harald Luessen was seen penning the following ode to ... whatever:
> > What everybody in the thread did first was to show the corresponding
> > loop, a three-liner, and nobody asked about its meaning. Everybody
> > understood immediately. Imagine the not so skilled collegues who
> > may have to understand and change your code.
>
> While this is something that needs to be kept in mind, you should also
> consider where you draw the line - do you "dumb down" the C++ experts
> in your team to the level of the weakest/least experienced member, do
> you require a minimum level of knowledge instead or do you "force" the
> less experienced members of the team to catch up with the gurus?

  Having taught C++ I believe that

    nail = find_if(haystack.begin(), haystack.end(), isNail);

  is far easier to understand for beginners than

    for (nail = haystack.begin(); nail != haystack.end(), ++nail)
      if (isNail(*nail))
        break;

  is. My reasoning:

  The name 'find_if' immediately reveals the
  meaning of this piece of code without any need
  to look at its implementation, while the loop
  shows the code's meaning /only/ through the
  implementation, which needs to be understood.

  Not incidentally, the 'find_if()' example uses
  only seven identifiers and 3 operators ('=',
  '(', ')', and ',') -- all of which are well
  known outside the realm of C++ coding and used
  there with the same meaning. The whole thing is
  an assignment with a function call on the right
  -- one of the most fundamental constructs of
  many programming languages which is usually
  taught during the very first lesson.
  The loop, OTOH, employs three keywords, nine
  identifiers, and three more operators ('!=',
  '++', and '*') -- all of which have no meaning
  or a different meaning for non-(C++)coders.
  It's a branching statement inside a looping
  statement which is terminated in the middle
  -- not the simplest of language constructs.

  I conclude:
  Just because after many years of coding most
  coders are used to decoding loop constructs
  without even thinking of it, that doesn't
  mean they really /are/ easier to understand.
  It only means that they have learnt to do it.

> [...]

  Schobi

-- 
SpamTrap@gmx.de is never read
I'm Schobi at suespammers dot org

"Coming back to where you started is not the same as never leaving"
 Terry Pratchett



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Dietmar Kuehl wrote:
> Mirek Fidler wrote:
> 
>>Dietmar Kuehl wrote:
>>
>>>Actually, it is easy to create a container similar to vector which
>>>does not suffer from invalidating iterators while mutating the
>>>contents: the iterators would simply store a pointer to the container
>>>and an index. In fact, you can even take 'std::vector' for this but
>>>use different than the normal iterators...
>>
>>Sure. But it solves only invalidating of iterator, not out-of range
>>problem.
> 
> 
> Are you confident that it is impossible to solve the out-of-range
> problem the same way, e.g. by including appropriate checks with the
> corresponding operations? There are many other things you can put

Well, general algorithm check will be it != foo.end(). There is not much 
you can do there unless you force random access iterator requirement to 
algorithm - but then you can use index as easily...

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Sorry Ctrl+Enter doesn't create a new line in this editor, it sends the 
message...

"Mirek Fidler" <cxl@volny.cz> skrev i meddelandet 
news:3ku3i9Fvql7tU1@individual.net...
>> | Sure. But it solves only invalidating of iterator, not out-of range 
>> problem.
>>
>> Isn't that one already solved?
>
> Well, I am addressing issue when some elements from container are
> removed while performing loop, compare
>
> for(std::vector<foo>::iterator it = bar.begin(); it != bar.end(); 
> ++it)
>     if(condition(*it))
>        bar.erase(it);

Which can also be written as

bar.erase(std::remove_if(bar.begin(), bar.end(), condition),
                   bar.end());

and save you a lot of copying, if you are removing more than one 
element.

>
> for(int i = 0; i < bar.GetCount(); i++)
>     if(condition(bar[i]))
>        bar.Remove(i);
>
> (not that above code would be useful, but I hope it helps to 
> illustrate
> the problem).

Exactly, the index is still "useful" in that it is a valid index into 
the vector, just not the same element. So you just created another 
problem instead. :-)


The original "problem" can of course be fixed, by recognizing that 
bar.erase(it) returns an iterator that are still valid. Your code throws 
the valid iterator away, and then you complain that your iterator is 
invalid!


Bo Persson



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Mirek Fidler" <cxl@volny.cz> skrev i meddelandet 
news:3ku3i9Fvql7tU1@individual.net...
>> | Sure. But it solves only invalidating of iterator, not out-of range 
>> problem.
>>
>> Isn't that one already solved?
>
> Well, I am addressing issue when some elements from container are
> removed while performing loop, compare
>
> for(std::vector<foo>::iterator it = bar.begin(); it != bar.end(); 
> ++it)
>     if(condition(*it))
>        bar.erase(it);

Which can also be written as

std::erase(std::remove_if(bar.begin(), bar.end(), condition),

>
> for(int i = 0; i < bar.GetCount(); i++)
>     if(condition(bar[i]))
>        bar.Remove(i);
>
> (not that above code would be useful, but I hope it helps to 
> illustrate
> the problem).
>
> Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Dietmar Kuehl wrote:
> Mirek Fidler wrote:
> 
>>>I stick to what I said. In fact, I would even go a step further: the
>>>real contribution of STL is entirely immaterial in C++!
>>
>>Hehe, I think we can agree on that :)
> 
> 
> Note the difference between "irrelevant" and "immaterial". What I
> mean is that the real contribution of STL, i.e. the concepts, are
> not really expressed in [current] C++ source code at all.

I think I have understand you well (this time :). I agree with your 
point anyway.

>>That is why I have started thinking about swithing to some different
>>approach, perhaps based on some general container interface. In the end,
>>you can always transform begin - end pair to the container
>>representation....
> 
> 
> A containers interface is the wrong level of abstraction, at least
> for my uses. Many of the sequences I process are not directly container
> and e.g. do not allow insertion or removal of elements.

Well, first of all, I perhaps should not have started speaking about my 
current "working hypthesis", it is completely off-topic.

Anyway, having said that, I think that completely depends on how such 
'container representation' would look like...

> It is definitely not a myth in the area of graph algorithms where it
> is quite conventional to insert and remove e.g. auxiliary edges on
> the fly. The position to insert or remove at is naturally obtained
> from the current position which in turn is e.g. immediately, without
> traversing the list at all, obtained from an entering edge.

Well, perhaps problem is that I am still speaking about STL and how its 
requirements seem to be inferior for application development.

I never ever tried to criticize e.g. using of templates, providing 
container templates etc..

>>I am afraid that number of real world problems where list has any
>>advantage is pretty close to zero. I have not found any real use for
>>list container since I have started using C++ (for 12 years).
> 
> 
> Maybe this is due to your view? My view is different and personally
> I have put lists to good use in the past (although these had not
> necessarily been 'std::list' but some form of circular list).

Well, I am using circular lists all over again.

>>a) my algorithms allow T to have implicit move copy (a = b; - b can now
>>be "moved", well, in my terminology, "picked")
>>
>>b) sort requires just iter_swap and compare predicate.
> 
> 
> To meet these requirements I can only envision the following
> alternatives for your containers:
> 
> - They are always based on node based structures (e.g. each element is
>    actually represented by a pointer to the element in the container).
>    For the allocation of nodes exists different alternatives, in
>    particular, it is not necessary to allocate each node individually.
>    However, this defeats certain deliberate goals like colocation of
>    objects, e.g. to fit into only few cache lines.
> - You can only operate on PODs with the obvious and known restrictions.
> - You use non-portable approaches like memcopy()ing non-POD objects.

Well, I would like to direct attention to the fact that I was speaking 
about algorithm requirements, not about containers, mostly as 
explanation why I had to implement my version instead using that 
provided with standard library?

(And yes, they do it all :)

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

David Abrahams wrote:
> Mirek Fidler <cxl@volny.cz> writes:
> 
> 
>>That is why I have started thinking about swithing to some different 
>>approach, perhaps based on some general container interface. In the end, 
>>you can always transform begin - end pair to the container 
>>representation....
> 
> 
> In my current work I am working with a "Sequence" concept (yes, I know
> the standard has one but it's pretty weak and I needed a name) that
> bundles a property map and two cursors.  That seems to be simplifying
> the interface and improving composability.
> 

Well, that sounds like a place where I am heading now.... :)

Some code to observe somewhere? :)

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Gene Bushuyev wrote:
>Let me just add, that learning STL is a very good way of learning C++
>generic programming.

Never for beginners

>...
>So if a person doesn't understand most part of STL then he probably missed
>a hole lot of C++. Just a thought.

I have no opinion how one should or should not like/use STL.
But I am also curious what's the lot part of programming one
can miss if not using STL? That is interesting.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> That is very interesting.  
> Very recently, I was asked the question "do you know of a genuine use
> of (linked) list that is simple to explain?"  Since the question was
> asked by someone who obvisouly is not what is called a beginner, I
> paused a bit before risking my first tentative (which turned out to be
> "wrong" anyway).  Then, we spent a couple of days before reaching some
> kind of agreements on something.  The key observation that killed most
> of the examples was that each time one believes that a list is
> needed/suitable, actually one was thinking of the abstract notion of
> "sequence" or "list", not the "linked list" data structure.  Usually
> the items are part of other bigger data structures and are chained
> together by conceptual links and similar things. And a better and simpler
> representation always seemed to exist.

This is exactly my long term observation, something I wanted to express 
with my far inferior english.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <MPG.1d535315dd12a43e9897e1@news.hevanet.com>, Scott Meyers 
<Usenet@aristeia.com> writes
>In summary, the empirical data suggest that there is no noticable
>advantage of algorithm calls over for-loops with regards to
>comprehention speed nor with regards to recognizing the purpose
>of a stereotypical loop.

Really? What about std::sort(), std::stable_sort() etc. The big strength 
of algorithms is that such simple things as find_if() really aren't what 
they are about. Indeed I would question whether such things are 
algorithms in any meaningful sense.

But once we start using algorithms (and adding our own) we might as well 
get used to the STL style instead of having two different ones and 
having to decide which to use in each case.

I am not advocating that we automatically try to STLize for loops, but I 
am advocating that when we want to do something already provided or when 
we determine that we have something that we want to do generically we 
consider going the STL way.

BTW, I tend to think that all the provided STL components in the 
Standard Library are just a starter pack. The true strength of the STL 
concept is its extensibility. If we can provide better next time, fine. 
But note that is always true. Taking the first step is the big leap.

>

-- 
Francis Glassborow      ACCU
Author of 'You Can Do It!' see http://www.spellen.org/youcandoit
For project ideas and contributions: http://www.spellen.org/youcandoit/projects


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <m364utrimr.fsf@uniton.integrable-solutions.net>, Gabriel Dos 
Reis <gdr@integrable-solutions.net> writes
>Indeed, I'm wondering why std::vector would not have been a better
>representation in terms of simplicity and efficiency.

One of the interesting observations mad several years ago by Andy Koenig 
was that whilst std::list seemed to have the edge over std::vector 
because of the 'ease' with which items could be inserted and deleted 
that may be based on theory rather than practice.

He then went on to point out that with modern multi-level cache systems, 
the contiguous nature of the vector's data storage can result in a major 
performance gain, even an insert may be faster for small to medium size 
data structures.

That leaves me wondering whether decisions to use std::list are not 
often cases of premature optimisation.

I suspect that std::list is only actually useful in programs that are 
handling multiple lists with splicing between them and slicing of 
individual ones. For example, writing an implementation of Lisp in C++ 
might be a good use of std::list.


-- 
Francis Glassborow      ACCU
Author of 'You Can Do It!' see http://www.spellen.org/youcandoit
For project ideas and contributions: http://www.spellen.org/youcandoit/projects


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <1122609899.216027.162800@f14g2000cwb.googlegroups.com>,
 "Bob Bell" <belvis@pacbell.net> wrote:

> Howard Hinnant wrote:
> 
> [snip code example]
> 
> > 1.  vector<ostringstream> !!!
> > 2.  No max_dump_index.
> > 3.  The ostringstream is manipulated both outside of, and inside of the
> > container.  ostringstream state is transferred into the container.
> > 4.  std::remove_if is used to manipulate the vector<ostringstream>.
> > 5.  ostringstream is still neither CopyConstructible nor CopyAssignable.
> > 6.  Syntax is nearly indistinguishable from today's syntax.  There is
> > very little new to learn here.  Things just work.
> > 7.  ostringstream is not special.  You can do the same thing with your
> > movable but non-copyable types.  Just give them a move constructor and
> > move assignment (typically less difficult than coding a copy constructor
> > and copy assignment).
> 
> Pretty darn cool!
> 
> Question: what happens if you use "os" after it's moved?
> 
>     for (int i = 0; i < n; ++i)
>     {
>         std::ostringstream os;
>         os << (i+1) << " : ";
>         dumpfiles.push_back(std::move(os));
> 
>         os << "hello";
>     }

After std::move(os), os is in an implementation defined state.  You 
should be able to destruct it (by present day rules), or "assign" it a 
new value, which in this case would involve str() and perhaps clear().

The general principle is that you can move from rvalues, but not lvalues:

    dumpfiles.push_back(std::ostringstream("some string"));

But you can also override the restriction on moving from lvalues with 
the std::move(os) function.  By doing so, you're telling the compiler to 
treat os as if it were an rvalue, and thus you really don't care what 
its value is after the expression.  If you really do care, then using 
move in this place would be a programming error.

In reviewing the Freescale std::lib code, I've been impressed with how 
often I've used "copy" when my intent was really "move".  A trivial 
example is:

template <class T>
void
swap(T& a, T& b)
{
    T tmp(a);
    a = b;
    b = tmp;
}

In each of these three statements, I really don't care about the value 
of the expression on the rhs after the "copy" because in the very next 
statement I'm going to give that variable a new value.  Therefore the 
following reimplementation has the same logic, but is potentially 
faster, and no longer requires CopyConstructible and CopyAssignable 
(only MoveConstructible and MoveAssignable):

template <class T>
void
swap(T& a, T& b)
{
    T tmp(std::move(a));
    a = std::move(b);
    b = std::move(tmp);
}

-Howard

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <3kujf1F10acgmU1@individual.net>,
 Mirek Fidler <cxl@volny.cz> wrote:

> The main problem with Howard's approach is that it does work only with 
> elements with specific interface and does not solve polymorphic issues.

This is true (at least the part about polymorphism).  Containers still 
hold by value, not by reference or pointer.  So you can still not:

vector<abstract_base_class> v;

And if you do:

vector<base_class> v;

you still have slicing issues.  I have thought about:

vector<base_class&> v;

but am unsure if it is really a good idea.

vector<shared_ptr<base_class> > v;

is still a good idea.  However I am proposing a variant to complement 
(not compete with) shared_ptr:

vector<unique_ptr<base_class> > v;

unique_ptr<T> is very similar to auto_ptr<T>, having the same overhead 
and nearly identical functionality.  The most important difference 
between unique_ptr and auto_ptr is that unique_ptr will not move from 
lvalues, only from rvalues (auto_ptr moves from lvalues and rvalues).

unique_ptr<int> p1;
unique_ptr<int> p2(p1);  // compile time error
unique_ptr<int> p3(std::move(p1));  // ok, p1 moved to p3

This difference is key to making unique_ptr safe to put into containers, 
and safe to use with generic algorithms.  If the container or algorithm 
attempts to copy from an lvalue, it will fail at compile time.  If the 
container or algorithm judiciously uses std::move (casting lvalues to 
rvalues), then the code (instantiated on unique_ptr) will just work.

vector<unique_ptr<T> > will have lower overhead, and be faster than 
vector<shared_ptr<T> >.  However it has different semantics that may or 
may not be what the client requires.

vector<unique_ptr<T> > enforces at compile time that each heap based 
pointer is owned by exactly one smart pointer at all times.  This means 
that the container of unique_ptr is not copy constructible nor copy 
assignable (though the container is movable, and can even be returned by 
value from functions - cheaply!).

vector<shared_ptr<T> > allows shared ownership.  If shared ownership is 
what is required of the client, then this is the way to go.  However if 
the only reason you're using shared_ptr is just to hold the pointer in 
the container, then using unique_ptr is likely to be more efficient.

-Howard

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <1122575551.396897.131000@z14g2000cwz.googlegroups.com>,
 "Allan W" <allan_w@my-dejanews.com> wrote:

> Howard Hinnant wrote:
> > The rvalue reference proposal - library recommendations (
> > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1771.html )
> > addresses algorithms as well as the containers.  Check out the
> > number of  mutating algorithms listed in there that do not require
> > CopyConstructible or CopyAssignable.  They include:
> 
> Excuse me for responding without having first read the library
> proposal that you linked to first, but...
> 
> > remove
> > remove_if
> > unique
> > reverse
> > rotate
> > random_shuffle
> > partition
> > stable_partition
> > sort
> > stable_sort
> > partial_sort
> > nth_element
> > inplace_merge
> > the heap algorithms
> > the permutation algorithms
> 
> ...but how do you implement any of these... especially the shuffles and
> sorts... without being able to move the elements from one location to
> another?
> 
> The answer has to be std::swap, right? But the general implementation
> of
> swap requires T to be assignable.

swap can be part of the solution, depending upon the specific algorithm. 
But it is not the entire solution (depending only on swap would be too 
inefficient in some circumstances).  See my other post in this thread 
where I show an improved implementation of the generic std::swap which 
no longer requires CopyConstructible or CopyAssignable.

The paper(s) introduce a new type of reference called the "rvalue 
reference".  Unlike the existing reference (now termed the lvalue 
reference), the (non-const) rvalue reference allows binding to an rvalue:

struct A {};

void foo(A&&);  // parameter is an rvalue reference to A

void bar()
{
    foo(A());  // ok, binds rvalue A to non-const A&&
}

Given this tool, and carefully chosen overload resolution rules, we can 
now overload foo() such that one overload sees lvalues, while the other 
sees rvalues:

void foo(const A&);  // lvalues (and const rvalues) bind here
void foo(A&&);       // non-const rvalues bind here

The second overload, knowing that it has bound to an rvalue A, can do 
whatever it wants to that rvalue, safe in the knowledge that the rest of 
the program won't notice.  Specifically, if A has resources (such as a 
heap pointer), foo can safely transfer ownership of those resources away 
from the A (as long as A's invariants are preserved).

Copy constructors and assignment operators can be overloaded in this 
fashion, giving types such as A "move constructors" and "move 
assignment" operators:

struct A
{
    A(const A&);  // copy constructor
    A(A&&);       // move constructor
    A& operator=(const A&);  // copy assignment
    A& operator=(A&&);       // move assignment
};

If A is expensive to copy, say because of needing to duplicate resources 
such as heap memory, it can usually be made cheap to move, by simply 
transferring those resources from the rvalue argument.

> If you aren't trying to prove some point, would you ever create a class
> that can be swapped but not copied or assigned? Have you ever done this?

Yes to both questions.  I have made the all of the std file streams, and 
string streams swappable, while they remained uncopyable (either in 
construction or assignment).  However these classes are now move 
constructible and move assignable (as well as swappable).  This results 
in them being "containerable" and usable with many more algorithms.

See example algorithm implementation here:

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2005/n1771.html#insert
ion_sort

-Howard

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> The main problem with Howard's approach is that it does work only with
> elements with specific interface

How is that different/worse than std::vector only working with elements
with a specific interface?

> and does not solve polymorphic issues.

It's this capacity (and the fact that your examples show the container
creating its containees) that makes me think that your Array<T> type
must be a dynamic array of pointers to T, even though the pointer is
hidden from the user. If that isn't how it's implemented, could you say
something about how it is? Can you add an already-constructed object to
an Array<T>?

Bob


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Francis Glassborow <francis@robinton.demon.co.uk> writes:

> For example, writing an implementation of Lisp in C++ might be a
> good use of std::list.

Although slist� might be more appropriate for approximating
singly-linked cons cells.


Footnotes: 
� http://www.sgi.com/tech/stl/Slist.html

-- 
Steven E. Harris

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

>>The main problem with Howard's approach is that it does work only with 
>>elements with specific interface and does not solve polymorphic issues.
> 
> 
> This is true (at least the part about polymorphism).  Containers still 
> hold by value, not by reference or pointer.  So you can still not:
> 
> vector<abstract_base_class> v;
> 
> And if you do:
> 
> vector<base_class> v;
> 
> you still have slicing issues.  I have thought about:
> 
> vector<base_class&> v;
> 
> but am unsure if it is really a good idea.

Well, Howard, what is wrong with Array way? (Apart from totally strange 
interface for STL folks).

I believe it does everything you want from such thing.

> vector<shared_ptr<T> > allows shared ownership.  If shared ownership is 

off-topic: shared ownership is IME a way to hell.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> It's this capacity (and the fact that your examples show the container
> creating its containees) that makes me think that your Array<T> type
> must be a dynamic array of pointers to T, even though the pointer is
> hidden from the user. If that isn't how it's implemented, could you say

Sure. I is.

> something about how it is? Can you add an already-constructed object to
> an Array<T>?

Yes. They are defined in methods description:

http://upp.sourceforge.net/src$Array$en-us.html

Add methods are first methods descibed, so I think it is not much 
trouble to get this information from there.

BTW, note that "AddPick" method is rather close equivalent to what 
Howard does (I mean .push_back(std::move(os))), but of course, using it 
with Array is less likely, closer equivalent to Howards code would be 
with Vector. (described here:

http://upp.sourceforge.net/src$Vector$en-us.html)

Please note one unfortunate terminology difference: What Howard calls 
"movable" is "pick" in my code and my "moveable" means that type can by 
memcpyied (yes, I know, this is breaking rules, but is far worth it).

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Dietmar Kuehl wrote:
> Scott Meyers wrote:
>
> - For many typical cases the algorithms are relatively hard to use due
>    to their need for pairs of iterators. Although the flexibility to
>    operate on subsequences is essential for a fundamental layer, it
>    makes their quite hard and unnecessarily wordy. It should be
>    possible to pass a "range" which exposes a 'begin()' and an 'end()'
>    (i.e. containers happen to be range objects) in addition to overloads
>    taking sequences. Although these algorithms would mere extract the
>    sequence and forward it to a corresponding algorithm, this has a huge
>    impact on usability and also allows algorithm chaining, i.e. passing
>    the result of one algorithm immediately as the input of another
>    algorithm.

Could you give us an example on how you envision the syntax for this to
be?

thank you,
- Anand


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Hendrik Schober wrote:

>   Saying "I know C++, but I don't know the STL"
>   for me sounds the same as saying "I know C++,
>   but I don't know about OO".

Actually, it is far worse.

It is like saying "I have never been to Europe, except for last year,
when I went to Europe"  (IOW, a self-contradictory statement).

Not knowing the STL restricts the first part of the statement to "I
know *parts* of the C++ language" at the most.  The statement "I know
C++" implicitly says that C++ is known in its entirety, which is then
contradicted by a statement that says that there is a part of the C++
language that is not known.

Carlos
--

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

<wij@seed.net.tw> wrote in message 
news:1122643038.414801.187940@g47g2000cwa.googlegroups.com...
> Gene Bushuyev wrote:
>>Let me just add, that learning STL is a very good way of learning C++
>>generic programming.
>
> Never for beginners

I don't know what your "beginner" means. Everybody who hasn't finished 
studying the basic language concepts and facilities is a beginner. Generic 
programming is a big part of STL. It doesn't present any extreem 
intellectual difficulties in understanding and memorizing the concepts and 
methods, it does require some time though. Difficulty comes when a practical 
knowledge of algorithms and principles of the sound design are required. 
Problem solving abilities do not simply come from knowledge of language.

>
>>...
>>So if a person doesn't understand most part of STL then he probably missed
>>a hole lot of C++. Just a thought.
>
> I have no opinion how one should or should not like/use STL.
> But I am also curious what's the lot part of programming one
> can miss if not using STL? That is interesting.

There may be reasons to avoid some parts of STL, but generally speaking it's 
a free lunch. All people I've met so far who shunned STL under the pretext 
(choose your favorite) were simply illiterate in C++. I would also be 
surprised to see somebody versed in boost or other advanced generic 
libraries and having no knowledge of STL.

- gene 


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <3kv3nmF107u58U1@individual.net>,
 Mirek Fidler <cxl@volny.cz> wrote:

> Well, Howard, what is wrong with Array way? (Apart from totally strange 
> interface for STL folks).

You are much more familiar with STL than I am with NTL.  I'm not 
qualified to comment much on it.  I complement you on creating a tool 
with the current language that has served yours and others needs so well.

While our work obviously has overlaps, and I think we are both motivated 
by the same problems, my goal is to make the existing std::lib work 
better while minimizing disruption to existing code, not create a 
completely new tool.

-Howard

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Gene Bushuyev wrote:
>There may be reasons to avoid some parts of STL, but generally speaking it's
>a free lunch. All people I've met so far who shunned STL under the pretext
>(choose your favorite) were simply illiterate in C++. I would also be
>surprised to see somebody versed in boost or other advanced generic
>libraries and having no knowledge of STL.

Almost all programming tasks I encountered were finished without
using STL. I know many people's tasks can't, simply because only STL
available for many.

STL but a Template Library among numerous ones, implemented by so
called 'advanced macro'.

So "I am also curious what's the lot part of programming one can miss
if not using STL? That is interesting.", the contrary might the truth.

Relatively few people is familiar with algorithmics and some
elementary programming design elements. These are provided handy
in STL.

The thing I know about the necessity of STL is not related to C++ but
stream/formated I/O, locale...


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:

> Well, I am addressing issue when some elements from container are
> removed while performing loop, compare
>
> for(std::vector<foo>::iterator it = bar.begin(); it != bar.end(); ++it)
>      if(condition(*it))
>         bar.erase(it);
>
> for(int i = 0; i < bar.GetCount(); i++)
>      if(condition(bar[i]))
>         bar.Remove(i);

Both loops are a stupid idea in the first place since the will miss
elements meeting the condition after removed elements. This is easily
corrected with something like this, however:

   for (some_iterator it = bar.begin(), end = bar.end(); it != end; )
      if (condition(*it))
         it = bar.erase(it);
      else
         ++it;

I'd guess that the index approach to the same problem looks actually
quite similar:

   for (int i = 0, size = bar.size(); i < size; )
     if (condition(bar[i]))
       bar.erase(i);
     else
       ++i;

In both cases you need to know what you are doing. No big difference
in my opinion although the iterator approach is actually more general
in that it can cope with containers like lists (of course, this is no
argument to you as you don't need containers like this; it is, however,
to me).
--
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
>> To meet these requirements I can only envision the following
>> alternatives for your containers:
>>
>> - They are always based on node based structures (e.g. each element is
>>    actually represented by a pointer to the element in the container).
>>    For the allocation of nodes exists different alternatives, in
>>    particular, it is not necessary to allocate each node individually.
>>    However, this defeats certain deliberate goals like colocation of
>>    objects, e.g. to fit into only few cache lines.
>> - You can only operate on PODs with the obvious and known restrictions.
>> - You use non-portable approaches like memcopy()ing non-POD objects.
>
> Well, I would like to direct attention to the fact that I was speaking
> about algorithm requirements, not about containers, mostly as
> explanation why I had to implement my version instead using that
> provided with standard library?

With no requirement whatsoever on elements held in a sequence, the
support for moving elements has to come from a different place. This
can only be the container (well, "container" should have been spelled
"sequence" but I wrote the article pretty late in the night).

> (And yes, they do it all :)

Note, that I consider all of these Bad Things which are restrictive at
best and easily range up to being non-portable. While I can live with
some restrictions, non-portable software is entirely unacceptable to
me, i.e. I stay within the guarantees of the standard. Where I do not,
I consider it an error which is fixed when I detect it. Not being
perfect, I may be ignorant about some problems but I never accept
deliberate violations like memcopy()ing non-PODs.
--
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Gabriel Dos Reis wrote:
> Dietmar Kuehl <dietmar_kuehl@yahoo.com> writes:
> | However, why focus on containers anyway? Scott specifically asked
> | about the *algorithms*!
>
> You're quite right.  However, I'd observe that we're divergeing
> talking about containers because algorithms need to work on
> "something" and that "something" usually is a container :-)

I'd say the reason is quite different. First of all, algorithms work
on sequences and, at least for me, it is quite usual that they work
on some form a view of something which may indeed be a container. It
could be a database query, too. Dealing with the details of containers
and their requirements, especially the restrictions with respect to
copy construction and assignment, is entirely irrelevant to algorithms.
I think the reason people keep coming back to containers is that it is
easier to nitpick about concrete container details than thinking about
the general concepts algorithms are based upon. However, it is more
important to get the latter right than discussing minor details of
specific containers.
--
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Francis Glassborow wrote:
> In article <MPG.1d535315dd12a43e9897e1@news.hevanet.com>, Scott Meyers
> <Usenet@aristeia.com> writes

>>In summary, the empirical data suggest that there is no
>>noticable advantage of algorithm calls over for-loops with
>>regards to comprehention speed nor with regards to recognizing
>>the purpose of a stereotypical loop.

> Really? What about std::sort(), std::stable_sort() etc.  The
> big strength of algorithms is that such simple things as
> find_if() really aren't what they are about.  Indeed I would
> question whether such things are algorithms in any meaningful
> sense.

He said "stereotyped loops".  I don't think that sort is
stereotyped in any programmers mind.  In an earlier post, I
mentionned the fact that the advantage of find_if over the loop
was slight; in the case of lower_bound, it was far greater.

I think, too, that there is an issue which has to be considered:
what is stereotyped.  The example given for find_if used break,
which is definitly not the way I would write it (see the SESE
thread).  I recognize simple loops which follow my stereotype
very quickly, but would you -- or Andrei, who definitly has a
different style in this than I do.  My impression is that my
stereotype is his abherition, or unusual case, and vice versa.

This is where the standard comes in: it provides a standardized
stereotype.  And for user defined fonctions, choosing a good
name provides something that can be universally agreed upon --
Andrei and I may not agree on how to write the loop, but we do
agree for the most part on the meaning of basic English words.

I've not seen the full study, to know exactly what was being
measured, but it seems intuitively obvious to me that
programmers with radically different styles but the same native
language will recognize well chosen function names better, where
as programmers with a homogenous style, but different native
languages, may be better off with the loops.  (My first question
concerning the study would be whether the programmers were
evalutating their own code, code written by someone whose style
they were familiar with, or code written by someone whose code
they were seeing for the first time.)

-- 
James Kanze                                 mailto: james.kanze@free.fr
Conseils en informatique orient�e objet/
                         Beratung in objektorientierter Datenverarbeitung
9 pl. Pierre S�mard, 78210 St.-Cyr-l'�cole, France +33 (0)1 30 23 00 34

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Hi,

Scott Meyers napisa�(a):
> [...]

That's a very interesting and provocative post from the reader of your 
book. I also a have similar and a little confusing thoughts.
After I've read many C++ books I see new features and techniques comming 
with C++ as a very seful and making C++ really multiparadigm language.

> The first question, especially, should really be answered only by people
> who spend a lot of time working on real C++ systems.  I welcome comments on
> both questions, as I hope to be able to enlighten not only my reader, but
> also myself.

Personally, I decided to move to STL algorithms usage but
I'm not trying to force myself to use it for every loop :-)
I'm programming applications for mobile devices - small apps for
small devices.
Some of my projects contains about 50-70 classes and I see them as "big 
systems" :-)
I started to use STL algorithms in my last 2 projects and first I 
decided replace simple loops (search, sort, update container elements) 
with for_each. I use also STL search and sort algo.

I my opinion for_each should be used instead of manually written loops
when functor/predicate can be short and simple.
IMHO, that makes source code clear and readable.
When my loop does something very complicated then I try to cut it down
to smaller pieces and then may be for_each fits somewhere.
But if loop should stay long and complicated then I don't recommend
for_each algo, because functor will be complicated too.
My two cents to discussion.
Best regards

-- 
Mateusz �oskot
mateusz at loskot dot net


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Howard Hinnant wrote:
> In article <3kv3nmF107u58U1@individual.net>,
>  Mirek Fidler <cxl@volny.cz> wrote:
>
>
>>Well, Howard, what is wrong with Array way? (Apart from totally strange
>>interface for STL folks).
>
>
> You are much more familiar with STL than I am with NTL.  I'm not
> qualified to comment much on it.  I complement you on creating a tool
> with the current language that has served yours and others needs so well.
>
> While our work obviously has overlaps, and I think we are both motivated
> by the same problems, my goal is to make the existing std::lib work
> better while minimizing disruption to existing code, not create a
> completely new tool.

That is completely understandable and it is the right way in your context.

Anyway, my point perhaps is that I would miss Array like container in
STL. For me it is so basic, obvious and essential that I really do not
understand why it is not included. And it would be as much new as
unique_ptr would.

Of course, I know that at the moment it is perhaps too late...

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Anand Hariharan wrote:
> Dietmar Kuehl wrote:
>> - For many typical cases the algorithms are relatively hard to use due
>>    to their need for pairs of iterators. Although the flexibility to
>>    operate on subsequences is essential for a fundamental layer, it
>>    makes their quite hard and unnecessarily wordy. It should be
>>    possible to pass a "range" which exposes a 'begin()' and an 'end()'
>>    (i.e. containers happen to be range objects) in addition to overloads
>>    taking sequences. Although these algorithms would mere extract the
>>    sequence and forward it to a corresponding algorithm, this has a huge
>>    impact on usability and also allows algorithm chaining, i.e. passing
>>    the result of one algorithm immediately as the input of another
>>    algorithm.
>
> Could you give us an example on how you envision the syntax for this to
> be?

This is nothing too exciting. For the purpose of this demo I will stick
to the usual iterators for brevity rather than splitting them into
cursors and property maps.

Now, the basic new concept then becomes a "range" consisting of
an object on which you can apply 'begin()' and 'end()' yielding
corresponding the well-known sequence delimited by iterators. Of
course, all STL containers are immediately ranges but something
like this would also be a valid range:

   template <typename InIt>
   struct range
   {
     typedef InIt iterator;
     typedef InIt const_iterator;

     range(InIt b, InIt e): m_begin(b), m_end(e) {}
     InIt begin() const { return m_begin; }
     InIt end() const   { return m_end; }
   private:
     InIt m_begin, m_end;
   };

For proper integration it is necessary to determine the exact type
of the iterators which cannot be done immediately through the range
because e.g. containers return different iterators depending on
constness. It could easily be done with some traits class, e.g.
like this:

   template <typename InIt>
   struct range_iterator<range<InIt> >
   {
     typedef InIt type;
   };

For containers there would be two versions of this class differing
in the constness of the type partially specialized on.

Given these definitions it is easy to define algorithms taking
ranges as arguments and simply delegating to corresponding algorithms.
For example, a 'find()' operating on a range could look like this:

   template <typename Range, typename Value>
   range<typename range_iterator<Range>::type>
   find(Range& r, Value const& val)
   {
     return range<typename range_iterator<Range>::type>(
       std::find(r.begin(), r.end(), val), // delegation
       r.end());                      // end of the auxiliary range
   }

Likewise, a version of 'copy()' taking a range for the input and an
iterator is before for the output could look like this:

   template <typename Range, typename OutIt>
   OutIt copy(Range const& r, OutIt to)
   {
     return std::copy(r.begin(), r.end(), to);
   }

Given these two algorithms you could use 'find()' to locate the start
of the copy and pass the corresponding range easily on:

   int main()
   {
     std::vector<int> v((std::istream_iterator<int>(std::cin)),
                        std::istream_iterator<int>());

     copy(find(v, 10), std::ostream_iterator<int>(std::cout, "\n"));
   }

This removes the need to litter code with '.begin()' and '.end()'
and avoids repeating access to the end iterator, too.
--
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Francis Glassborow wrote:
> In article <m364utrimr.fsf@uniton.integrable-solutions.net>,
> Gabriel Dos Reis <gdr@integrable-solutions.net> writes

>>Indeed, I'm wondering why std::vector would not have been a
>>better representation in terms of simplicity and efficiency.

> One of the interesting observations mad several years ago by
> Andy Koenig was that whilst std::list seemed to have the edge
> over std::vector because of the 'ease' with which items could
> be inserted and deleted that may be based on theory rather
> than practice.

> He then went on to point out that with modern multi-level
> cache systems, the contiguous nature of the vector's data
> storage can result in a major performance gain, even an insert
> may be faster for small to medium size data structures.

It depends.  If it is a list of
deep_copying_ptr<very_complex_class>, then a list can be faster
than a vector even with very few entries:-).

> That leaves me wondering whether decisions to use std::list
> are not often cases of premature optimisation.

Generally, the choice of std::list is based on the fact that
your code has iterators into the middle somewhere, and wants to
insert or delete there, without invalidating other iterators.
If your algorithm is keeping iterators for values, list has a
decided advantage over vector or deque.

It's not a question of optimization at all; it's a question of
program correctness.

> I suspect that std::list is only actually useful in programs
> that are handling multiple lists with splicing between them
> and slicing of individual ones.  For example, writing an
> implementation of Lisp in C++ might be a good use of
> std::list.

Actually, list is a must if 1) you add or remove elements from
the container, and 2) you maintain iterators (or pointers)
pointing to elements in the container.  (If you can establish an
ordering criteria, you could also use set, of course.  But list
allows modifying the elements it contains.)

--
James Kanze                                 mailto: james.kanze@free.fr
Conseils en informatique orient�e objet/
                         Beratung in objektorientierter Datenverarbeitung
9 pl. Pierre S�mard, 78210 St.-Cyr-l'�cole, France +33 (0)1 30 23 00 34

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Dietmar Kuehl wrote:
> Andrew Ward wrote:

>>Hendrik Schober wrote:

>>>  Er...because you do /not/ know how to program
>>>  in C++ unless you learned how to use the STL?

>>That is just plain wrong.

> Actually, I disagree: if you don't know the STL way, whether
> you need it or not, you don't know how to use all the C++
> potential.  Essentially, STL is an instance of a particular
> programming paradigm (Generic Programming) which is
> essentially only applied in the context of STL. Maybe
> Hendrik's statement is a little bit too strong, i.e. you can
> do effective work in C++ without knowing STL, but you
> definitely do not know all of C++ worth knowing.

The STL is vast.  You don't need to use or know generic
programming to adopt std::vector as your basic array type.

>>I would guess that in fact most C++ programmers do not use the STL.

> Ignorance has never been a good indicator for knowledge,
> especially if it was only accidental and not deliberate.

It's not only ignorance.  Until a year ago, our code had to work
with Sun CC 4.2 -- we didn't use the STL in it.

But I think that "most" above is an exageration -- most of the
job offers I see require STL experience, so someone is using it.

>>Think about the 100s of thousands of C++ programmers who use MFC,
>>ATL, VCL, Qt, CLI...

> It is an issue beyond what your are using. The basic ideas of
> most of these libraries are quite similar and rather different
> than those of STL, although some of these libraries have
> embraced at least some portions of STL. Essentially, I would
> claim that the foundation of all the libraries you mention and
> of most the libraries you didn't is object orientation. STL's
> foundation is an entirely different paradigm although it
> assumes some aspects of object orientation (e.g.

I'm not sure I agree.  Most of the low level libraries I've seen
embraced templates long ago.  And there's not much OO about
vector, regardless of whether it is the STL version or not.

Most of the other libraries are not as comprehensive or as
coherent as the STL.  It's not unusual to have a vector, a list
and a map class, all with iterators.  As far as I know, however,
only STL explicitly categorizes those iterators, and gives you a
means to determine the category in a template instantiation of
your own.  That doesn't mean that the other libraries are OO --
I would classify most of them as procedural.  And generic, but
with a more limited definition of "generic" than that used in
the STL.

>>All of these libraries provide alternatives to the STL

> Actually, they don't. They may provide containers as STL does
> but STL is not about containers.  STL is about algorithms
> which are entirely independent of containers.  They operate on
> sequences and containers happen to be examples of these
> sequences.  I don't think that any of the libraries you
> mention and most of the libraries you didn't provides an
> extensive set of algorithms.  Hence, they are no alternative
> to STL.

IMHO, the algorithms part is open in the STL.  The algorithms in
the standard library are good examples of what can be done, and
some of them are interesting in their own right, but part of the
point is that I can also furnish my own algorithms.  Algorithms
which, unlike those designed for another library, will work
indifferently on vectors, lists, or even istreams.

What sets the STL off from other libraries isn't any particular
bit of code.  It is the unity of concept.  Algorithms and
iterators aren't two independant concepts -- algorithms are
conceived in terms of iterators of a specific category, and
iterators are conceived to conform to a specific category.

>>and most were around long before the STL became standard. Just
>>because someone picks a third-party library and decides to
>>make it part of the C++ standard does not mean people have to
>>use it.

> Actually, this "someone" was a committee with all big players
> in the C++ area being represented and there was obviously a
> majority to pick this specific library and not others.

Were any others proposed?  Or was it a case of STL or no
general library?

-- 
James Kanze                                 mailto: james.kanze@free.fr
Conseils en informatique orient�e objet/
                        Beratung in objektorientierter Datenverarbeitung
9 pl. Pierre S�mard, 78210 St.-Cyr-l'�cole, France +33 (0)1 30 23 00 34

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

wij@seed.net.tw wrote:
> Gene Bushuyev wrote:

>>Let me just add, that learning STL is a very good way of
>>learning C++ generic programming.

> Never for beginners

I'm not sure that it's that good a way.  You don't need to know
any generic programming to use the STL.  And IMHO, you'd better
know it before starting if your goal is to implement an STL.

>>...
>>So if a person doesn't understand most part of STL then he
>>probably missed a hole lot of C++. Just a thought.

> I have no opinion how one should or should not like/use STL.
> But I am also curious what's the lot part of programming one
> can miss if not using STL? That is interesting.

The STL entails a lot.  std::vector is the standard array type
in C++; I would consider that someone who didn't use it didn't
know modern C++.  I'd also be a bit sceptical if someone
implemented a sorted map, rather than use std::map.  Or, for
that matter, wrote a loop when an STL algorithm did exactly what
was wanted.

I think the question here isn't so much whether you use the STL
at all, but where do you draw the line between using the STL and
writing your own loop.  Scott sort of presented it as a binary
choice, but it's really a continuum, and I doubt that there are
two programmers who will draw the line in exactly the same
place.

-- 
James Kanze                                 mailto: james.kanze@free.fr
Conseils en informatique orient�e objet/
                        Beratung in objektorientierter Datenverarbeitung
9 pl. Pierre S�mard, 78210 St.-Cyr-l'�cole, France +33 (0)1 30 23 00 34

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

>>>- For many typical cases the algorithms are relatively hard to use due
>>>   to their need for pairs of iterators. Although the flexibility to
>>>   operate on subsequences is essential for a fundamental layer, it
>>>   makes their quite hard and unnecessarily wordy. It should be
>>>   possible to pass a "range" which exposes a 'begin()' and an 'end()'
>>>   (i.e. containers happen to be range objects) in addition to overloads
>>>   taking sequences. Although these algorithms would mere extract the
>>>   sequence and forward it to a corresponding algorithm, this has a huge
>>>   impact on usability and also allows algorithm chaining, i.e. passing
>>>   the result of one algorithm immediately as the input of another
>>>   algorithm.
>>
>>Could you give us an example on how you envision the syntax for this to
>>be?
> 
> 
> This is nothing too exciting. For the purpose of this demo I will stick
> to the usual iterators for brevity rather than splitting them into
> cursors and property maps.
> 
> Now, the basic new concept then becomes a "range" consisting of
> an object on which you can apply 'begin()' and 'end()' yielding
> corresponding the well-known sequence delimited by iterators. Of
> course, all STL containers are immediately ranges but something
> like this would also be a valid range:
> 
>    template <typename InIt>
>    struct range
>    {
>      typedef InIt iterator;
>      typedef InIt const_iterator;
> 
>      range(InIt b, InIt e): m_begin(b), m_end(e) {}
>      InIt begin() const { return m_begin; }
>      InIt end() const   { return m_end; }
>    private:
>      InIt m_begin, m_end;
>    };
> 

Actually, this is exactly what I am now thinking about, but my working 
term was "generic container interface" (and I believe that this is what 
David Abrahams had in mind with "Sequence").

It is just that I would change this a little bit to replace iterator 
concept with "cursor" and added some more methods to such "range", like 
dereference of cursor etc... Of course, the 'range' based on iterators 
would be still trivial to realize.

One of motivations is fact that it is sometimes annoying that some 
iterators have to keep pointer to its container. This way this bloat 
would be avoided.

Other motivation is possibility to refine 'range' interface to be more 
effective than is allowed by regular iterators and iterator traits.

E.g. it would be nice to have "decomposed" iter_swap - to have interface 
entity for each part iter_swap - type for temporary variable and all 
three transfer operations.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> I consider it an error which is fixed when I detect it. Not being
> perfect, I may be ignorant about some problems but I never accept
> deliberate violations like memcopy()ing non-PODs.

Hm, perhaps you never used vector to store anything else than 
fundamental types?

That said, possibility that my code could run significantly faster by 
makeing one well-thought and safe standard violation just makes 
restless. Good for you that you can stay cool :)

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

James Kanze <kanze@none.news.free.fr> writes:

| > That leaves me wondering whether decisions to use std::list
| > are not often cases of premature optimisation.
| 
| Generally, the choice of std::list is based on the fact that
| your code has iterators into the middle somewhere, and wants to
| insert or delete there, without invalidating other iterators.
| If your algorithm is keeping iterators for values, list has a
| decided advantage over vector or deque.

I beg to differ.  It is not obvious that allocating the data somewhere 
else  and storing their addresses in a vector is not a better
alternative.  As I said, we spent a couple of days examining in
details examples based on the arguments you just developed (and
variants) and came to the conclusion that in the majority of the
cases, std::list did not seems really appropriate, optimization
notwithstanding.  The cases where  linked list seemed to be
appropriate were mostly complex data structures, but then the items
seemed better organized with instrusive lists.  I do know for sure we
have such codes running here and elsewhere.  Another example we
examined was the "pool allocator" idiom, as described in TC++PL3 for
example. 


-- 
                                                       Gabriel Dos Reis 
                                           gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Howard Hinnant wrote:
> In article <3kv3nmF107u58U1@individual.net>,
>  Mirek Fidler <cxl@volny.cz> wrote:
> 
> 
>>Well, Howard, what is wrong with Array way? (Apart from totally strange 
>>interface for STL folks).

Well, I guess, I found the answer - I am somewhat blinded by fact that 
in reality I am using just three basic types of containers all over 
again (well, I think that using anything else is useless, but that is 
completely another topic...). Maintaining 'polymorphic' and 'direct' 
versions is quite easy for me.

But you are right that in context of STL, providing unique_ptr will 
allow inserting polymorphic "values" to all containers, without the need 
to provide specialized "flavor" for doing so. In this context, wow! :)

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Dietmar Kuehl <dietmar_kuehl@yahoo.com> writes:

| I think the reason people keep coming back to containers is that it is
| easier to nitpick about concrete container details than thinking about
| the general concepts algorithms are based upon. However, it is more
| important to get the latter right than discussing minor details of
| specific containers.

In my case, the algorithms were uninteresting to me and the containers
where the core part -- I need to store my data somehwere before ever
thinking of fiddling with the bytes.  While there is some truth in
what you're saying I would be very hesitant to claim it is
representative of the situation.  And I think we should also resist
from the "empty data algorithm theory". :-)

-- 
                                                       Gabriel Dos Reis 
                                           gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"James Kanze" <kanze@none.news.free.fr> wrote in message 
news:42eb3d39$0$31035$636a15ce@news.free.fr...
....
>> Ignorance has never been a good indicator for knowledge,
>> especially if it was only accidental and not deliberate.
>
> It's not only ignorance.  Until a year ago, our code had to work
> with Sun CC 4.2 -- we didn't use the STL in it.
>
> But I think that "most" above is an exageration -- most of the
> job offers I see require STL experience, so someone is using it.

And this is a very interesting observation. When I interview people I ask 
questions about STL basic facilities, and if they don't know that, I know 
that they are weak in C++ in general. If I'm offered to take on a new 
project I'm asking if they were using STL, and if they didn't and there were 
no legitimate reasons why (and there are very few) I know that the project 
is probably a lump of spaghetti and I would definitely not want to take on 
it. If I'm asked to start a new project and someone insists I don't use STL, 
then I know that the project will probably be mismanaged, so it's better to 
steer clear. STL became such a convenient and reliable indicator.

- gene 


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Gabriel Dos Reis wrote:
> "White Wolf" <wolof@freemail.hu> writes:
>
>> In what way are indices more likely to stay valid than iterators?  I
>> mean if I have index #10 tored away for a vector, and then I remove
>> stuff and have only 5 elements, my index becomes rogue just the same
>> way an iterator would. I just do not see (or believe) that it is any
>> less likely with indices than iterators.
>
> push_back is most likely to invalid previous iterators than shrinking
> a vector.

That is why you will use indices with a vector (and not int, but size_t). 
But how does that prove that all iterators shall be banished and we should 
chanpe maps to hashes to satisfy this nonsense?  With a node-based container 
neither inserting, nor careful deleting will invalidate any active iterator.

-- 
WW aka Attila
:::
The Romans didn't find algebra very challenging, because X was always 10. 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Francis Glassborow wrote:
> In article <dc919p$94s$1@phys-news1.kolumbus.fi>, White Wolf
> <wolof@freemail.hu> writes
>> In what way are indices more likely to stay valid than iterators?  I
>> mean if I have index #10 tored away for a vector, and then I remove
>> stuff and have only 5 elements, my index becomes rogue just the same
>> way an iterator would. I just do not see (or believe) that it is any
>> less likely with indices than iterators.
>
> Very simply:
>
> if(index < v.size()) v[index] ...
>
> remains valid. IOWs the precondition for v[n] to be valid if v is a
> std::vector<type> is that v currently has at least n elements. There is
> an extra precondition for *v_iter to be valid; that the vector has not
> been moved since the value in v_iter was obtained. That pre-condition is
> not even checkable in the general case.

Vector.  Which is one special case, singled out.  But the statement by the 
one I replied to claimed that iterators has no right to exist, and we must 
use indices everywhere.

-- 
WW aka Attila
:::
When a cat is dropped, it always lands on its feet, and when toast is
dropped, it always lands with the buttered side facing down. I propose to
strap buttered toast to the back of a cat. The two will hover, spinning
inches above the ground. With a giant buttered cat array, a high-speed
monorail could easily link New York with Chicago. 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Stephen Howe wrote:
>> In what way are indices more likely to stay valid than iterators?
>
> If you have a vector that is full to capacity and you do a extra
> push_back() that normally invalidates any iterators as the underlying
> memory is reallocated. An index to an element before the push_back()'ed
> element is still valid.
>
> That is a big difference.

I am sorry, but can you give just *one* reasonable example when a programmer 
(in his right mind) would start to implement an iteration over a vector, 
while instering into it?

And let's - pretty please - not get carried away to the wrong (IMHO very 
wrong) direction of loosing sight of common sense!  See:

Node based container do not invalidate iterators during insertion, and 
*only* invalidate the deleted one during delete.  So in case you *must* use 
iterators, you don't have the problem above, so I find it amusing that it is 
attempted to be posted as a reason against using iterators.

In case of deque, there is no reallocation - it is the middle-way between 
vector-like and pure node-based ones.

In case of contiguous-memory-based, dynamically sized containers there is no 
need to use iterators for simple tasks.

But as I have indicated in another post: I cannot imagine any reasonable 
algorithm, which wants to iterate through a container (vector or string) and 
insert elements on the way.

And *if* such an algorithm has to be done:

- one will certainly use indices for a vector/string

- one will have no problem with node based containers

So I cannot accept the above as a proof that iterators must cease to exist. 
I could also list badly designed code to "prove" anything.

-- 
WW aka Attila
:::
The world is full of willing people -- some willing to work and some
willing to let them. - Robert Frost. 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

On 29 Jul 2005 07:07:48 -0400, Scott Meyers <Usenet@aristeia.com> posted
for Bjorn Reese:

> I recently conducted a study that, amongst others, investigates this
> claim. The study has not been published yet, as I am still analyzing
> the results.

> One of the results of the study showed that programmers

For us to understand this, we need a definition of programmer.  Is this
yet another psychology of the college student (freshman?) study?

John

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> Actually, list is a must if 1) you add or remove elements from
> the container, and 2) you maintain iterators (or pointers)
> pointing to elements in the container.

Well, but I will have just to see some real-world example where (2) is 
true (in "iterators" variant, of course. We are speaking about std::list 
here).

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
>> In conclusion, on a more modern compiler, i++ and ++i have no
>> performance
>
> Well, technically speaking, that is true in most cases, but OP is right
> that in generic iterator algorithm, using ++i is better, because in
> theory, ++ operators for some strange container iterators could be
> defined in a way that really makes i++ significantly slower.

It does not need to be significantly slower.  You are copying an object in a 
thight loop.  If the iterator is only a pointer, it does not take a lot of 
CPU.  BUT the loop itself might *also* only touch an int-sized thing!  You 
have just (successfully) increased the cost of *each* loop iteration(!) by a 
100%!  There is a reason for Duff's device.

Now let's assume, that your compiler has optimized the copying away.  In 
*optimized* code!  But since we have *pessimized* our code, it will not 
remove that additional copying in debugging/testing, making the loop to run 
half speed, burning more energy on the CPU, and possibly making the system 
slow enough so that external entities will time-out during testing.  All 
this for *nothing* else just for one's ego: But I cannot be wrong, I will 
even look at assembly code to show that I am not wrong.

-- 
WW aka Attila
:::
Stand for something, or you'll fall for anything. 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

E. Rosten wrote:
> On Wed, 27 Jul 2005 19:19:47 -0400, White Wolf wrote:
[SNIP]
> If you append stuff to a std::vector, iterators become invalid, indices
> do not.

I see no reason to append to a vector in an iterating loop.  Simply I cannot 
imagine an algorithm which would need that and would make sense.-

> [...]
>
>> 1.) ++it and *not* it++.  For gods sake: Mean what you say, and say what
>> you mean!
>
> I don't quite get what you mean here...

Do you want to say: step to the next element, or do you want to say: save me 
the value of i now, and then make it step to the next element?  Do you need 
pre or post increment here?  You need pre increment.  So say what you mean, 
and mean what you say.

>> 3.) Unless the container itself is being changed inside the loop, the
>> end() iterator is better be saved away into a const variable before the
>> loop.
>
> This is no longer true:
[SNIP]

Excuse me?  Do you seriously state, that "One shall pessimize his code 
because I saw one compiler, which does the right thing even then"?

> In conclusion, on a more modern compiler, i++ and ++i have no performance
> difference.

OK.  Now take a deep breath, and say it to yourself: this is not true. 
Nothing whatsoever (in the standard or otherwise) ensures your hope.  It is 
like running through a red-light, saying that "modern cars have ABS 
systems".  Nothing whatsoever entitles any programmer to write bad code 
which is just as easy to write as good code.  Just put the ++ before the 
variable name and stop arguing.  Just don't write such bad code.

> On even slightly less modern compilers, there is no need to
> use a temporary variable for .end(), since the compiler can figure it out
> for itself.

Really?  Could you please point out the chapter and verse from the C++ 
standard which requires the compiler to do so?

-- 
WW aka Attila
:::
The Romans didn't find algebra very challenging, because X was always 10. 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
[SNIP]
> Well, general algorithm check will be it != foo.end(). There is not much
> you can do there unless you force random access iterator requirement to
> algorithm - but then you can use index as easily...

I strongly disagree.  Two points:

1.) You are trying to hide programmer errors, which does not lead anywhere I 
want to go

2.) I still do not see *any* reason to change a map to a hash, just because 
I am afraid of iterators.

Point shortly: code which *gets* into the situation of out-of-range 
reference into a container is *faulty*.  The fact that its fault does not 
make the program to crash does not mean that it does the job it should do. 
I mean how do you expect that a programmer, who is unable to make his code 
stop at the end of a container without training-wheels will be able to 
implement the functionality of the loop at the same time?  How did he define 
his invariant, if he has no clue where his iterators/indices point???  All 
*my* loops have != even for indices.  If they do not stop, I *want* to know 
detect it in unit testing, and not hope that if I hmcked up the iteration my 
loop body will still magically happen to do its job.

-- 
WW aka Attila
:::
When a cat is dropped, it always lands on its feet, and when toast is
dropped, it always lands with the buttered side facing down. I propose to
strap buttered toast to the back of a cat. The two will hover, spinning
inches above the ground. With a giant buttered cat array, a high-speed
monorail could easily link New York with Chicago. 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Bo Persson wrote:
> "Mirek Fidler" <cxl@volny.cz> skrev i meddelandet
> news:3ku3i9Fvql7tU1@individual.net...
[SNIP]
>> for(std::vector<foo>::iterator it = bar.begin(); it != bar.end();
>> ++it)
>>     if(condition(*it))
>>        bar.erase(it);
>
> Which can also be written as
>
> std::erase(std::remove_if(bar.begin(), bar.end(), condition),
[SNIP]

Which, in fact, (together with Dietmar's observation that the hand-written 
loop was *faulty* (again (suprise, surprise)) proves my original point: you 
use algorithms *not*  because they are trendy or fancy, but because they 
make your code more robust and (in most cases) simple.  It does not mean 
that I use algorithms all the time.  But it also means that sometimes I do 
regret it, because I make all those mistakes as well.

-- 
WW aka Attila
:::
A real leader faces the music, even when he doesn't like the tune. 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
[SNIP]
>> Actually, it is easy to create a container similar to vector which
>> does not suffer from invalidating iterators while mutating the
>> contents: the iterators would simply store a pointer to the container
>> and an index. In fact, you can even take 'std::vector' for this but
>> use different than the normal iterators...
>
> Sure. But it solves only invalidating of iterator, not out-of range
> problem.

Nope, it does not even solve that!  It only ensures that you won't *crash*. 
Remember: an iterator, which suddenly starts to point to a different element 
(in case of an erase) is just as well invalid as one which points after the 
elements.

-- 
WW aka Attila
:::
Original Pentium of Borg: Division is futile - your decimals will be
approximated. 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Gabriel Dos Reis wrote:
[SNIP]
> Allowing vector<T>::iterator to be plain "T*" and allowing "T*" to be
> mapped directly to comon hardware thingy directly weaken the
> usuability of vector in terms of iterators.  For practical uses, I
> maintain a (container, index) pair.

And nothing whatsoever stop you from doing it.  And it *is* reasonable to do 
so, and to provide that new type as MyTable::iterator towards the user's (if 
you need to provide such thing).  But I still fail to see how does that 
(singled out, special case of vector) prove that claim that all iterators 
should disappear, and we should invent index-based maps.

-- 
WW aka Attila
:::
Question Authority. They usually know where the bathroom is. - MTV's Daria 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Dietmar Kuehl wrote:
> Gabriel Dos Reis wrote:
[SNIP]
>> push_back is most likely to invalid previous iterators than shrinking
>> a vector.
>
> Actually, it is easy to create a container similar to vector which
> does not suffer from invalidating iterators while mutating the
> contents: the iterators would simply store a pointer to the container
> and an index. In fact, you can even take 'std::vector' for this but
> use different than the normal iterators...

That is what my fixed-string iterator does.

-- 
WW aka Attila
:::
We occasionally stumble over the truth, but most of us pick ourselves up
and hurry on as if nothing happened. 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
>>> IME, and I know most people will disagree with me, the most effective
>>> solution is to avoid node base container and iterators and use indicies
>>> everywhere.
>>
>>
>> Good luck for having an indexed map. :-)
>
> Like this:
>
> http://upp.sourceforge.net/src$ArrayMap$en-us.html
>
> ?
>
> :)
>
> (And yes, it is faster than std::map for average case, before you start
> asking).

No, I do not start asking.  I am telling: that is not a map.  That is a 
hash.

>> In what way are indices more likely to stay valid than iterators?  I
>> mean if I have index #10 tored away for a vector, and then I remove
>> stuff and have only 5 elements, my index becomes rogue just the same
>> way an iterator would. I just do not see (or believe) that it is any
>> less likely with indices than iterators.
>
> Well, loops based on indicies have usually i < c.GetCount() (or i <
> size() if you insist) condition. That rules out most of problems.

???  And how?  If the container is changed inside the loop's body (and where 
else could you change it?) the above condition may just come way too late. 
It is only checked upon a new iteration.

> Usually, the body of loop performs some single action.
[SNIP]

In your codebase, maybe. :-)

> With indicies it usually works as expected out of box without trouble.

Well, ity work the same way for me with iterators.  And for people who 
refuse to think and design their code, there are other languages designed. 
:-)

> If you want example, go several news threads back and see "Vector
> Iterator headache". And I guess it is at least one post per week that
> deals with exactly the same problem.

Well, I think that the problem (in this case) is between the keyboard and 
the chair.

> And, BTW, performance wise, there is no difference either. Compiler will
> emit similary effective code regardless you are using indicies or
> iterators.

And who said there is?  What there is: added complexity (such as hash 
function) in the name of less complexity.  It sounds to me too much like 
some other language philosophy else under the Sun.

>> Of course, it *is* motivation for some people to "just do it" and make
>> code just to show off.  But IMHO most successful C++ programmers (still
>> being programmers and alive ones ;-) ) do not do that.  One uses a
>> language (or library) feature, because the design is expressed best
>> with that solution.
>
> Well, why it is "recommended style" then? That is I think OP's issue.

Who said it is recommended style?  Let's get our lines straight please. 
Wasn't it the OP's book, Effective STL, in Item 43 which says (in the title) 
"Prefer algorithm calls over hand-written loops"?

So let's clear it up: there is no C++ God (yes, ask Bjarne, he will say the 
same ;-) ).  Or C++ Bible which sets "recommended styles". (Even if some 
loves to claim their book it is - some who has never posted or read here. 
:-)

So we should not say "recommended style".  Let's say why Mr. X or Dr. Z 
recommends it.  I haven't read so far any accepted expert, who would have 
blindly recommended algoritms over loops (or vice versa) *without* telling 
the exact circumstances for which he thinks one should choose one over the 
other.

>> 2.) std::vector<std::string>::iterator is something, for which I flunk
>> pupils.  Over 40 ones.  Is it really necessary to repeate that
>> std::vector<std::string> in 254 places of the code?  How about having a
>>
>> typedef std::vector<std::string> str_table_t;
>>
>> inside that class having that very container as a member?
>
> Great. Now you had to introduce another type just to keep your broken
> machine going. Adding entities to code just to make your loops run does
> nothing than increase your code complexity.

???  I am sorry, but this is B/S.  Broken machine?  I am sorry, but the fact 
that *you* cannot use it does not make it broken.  But this "reasoning" 
sounds a whole lot like the broken record played by Java advocates for the 
past 15 years: pointers are evil, broken, bad...

You are making those typedefs, not because the language is broken, but 
because your desing is.  Repeating int instead of std::vector<std::string> 
only makes the repeated implementation detail shorter, but it does not 
remove it.

>> So if tomorrow I
>> decide to use list, or deque
>
> So what? How str_table_t is going to help you? Do you plan to replace
> ALL occurences of std::vector<std::string> in your code with list or
> deque? Or do you plan to introduce
>
> str_table_for_my_Foo_class_implementation_t
>
> like things?

Did you actually read what I wrote?  In my code, any code I write, any 
container is implementation detail *unless* I am implementing one.  The 
*only* code where I need to change anything, is the code of the class which 
implements the *real* interface to that table I need.  And it usually won't 
have *any* other functionality then implementing the table itself.  Which 
does *not* define its interface in terms of the container used internally, 
but in terms of the *requirements* of the *concept* it represents.

> Anyway, for me this is much more simple. If I want to replace one of my
> vector-like container with another, I just do that. They have similar
> interface and no additional types needed to write code with them (thanks
> to indicies).

Well, I have no trouble with iterators.  I have very well established rules 
of using containers.  One such rule is that *no* direct index or iterator 
for a container shall be stored away outside of the realm of one atomic 
algoritm.  Since those values are not defined in the terms of the user, but 
in the of the implementation.  What *may* be stored away is a *key*.  Or 
maybe a special type, which detects if its invalid - but I never needed to 
use this.  Of course, this is the case when the container might get updated. 
For static stuff, storing away fast-access index-values is OK, and in that 
case it matters not whether it is an iterator (for node based stuff) or an 
index value (size_t, not int - for the others).

-- 
WW aka Attila
:::
Hey! It compiles! Ship it! 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

White Wolf napisa�(a):
> [...]

White Wolf, the only thing I want to say is that I had
a great time reading your post, very funny but
certainly very instructive.
I agree with your arguments, they are the same I take
using STL algorithms.

Cheers

-- 
Mateusz �oskot
mateusz at loskot dot net


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

James Kanze wrote:
> Dietmar Kuehl wrote:
>> Actually, I disagree: if you don't know the STL way, whether
>> you need it or not, you don't know how to use all the C++
>> potential.  Essentially, STL is an instance of a particular
>> programming paradigm (Generic Programming) which is
>> essentially only applied in the context of STL. Maybe
>> Hendrik's statement is a little bit too strong, i.e. you can
>> do effective work in C++ without knowing STL, but you
>> definitely do not know all of C++ worth knowing.
>
> The STL is vast.  You don't need to use or know generic
> programming to adopt std::vector as your basic array type.

However, if you don't know about generic programming, you don't know
C++. It is like using a toolbox and deciding not to use the screw
driver because it is more complex than the hammer which can also be
used to insert screws into something.

>> It is an issue beyond what your are using. The basic ideas of
>> most of these libraries are quite similar and rather different
>> than those of STL, although some of these libraries have
>> embraced at least some portions of STL. Essentially, I would
>> claim that the foundation of all the libraries you mention and
>> of most the libraries you didn't is object orientation. STL's
>> foundation is an entirely different paradigm although it
>> assumes some aspects of object orientation (e.g.
>
> I'm not sure I agree.  Most of the low level libraries I've seen
> embraced templates long ago.

Embracing templates and Generic Programming are two rather different
things. Not everything which using templates qualifies as Generic
Programming - at least according to my view of Generic Programming.

> And there's not much OO about
> vector, regardless of whether it is the STL version or not.

If it uses inheritance to derive e.g. from "collection" or requires
its members to derive from "object" I'd see [inappropriate] use of
object orientation. ... and these are not just design decisions I
have made up: it is quite unusual that the containers are not related
in some way using inheritance or that you can plug arbitrary object
into them. There are other libraries than STL which relate containers
differently than the typical object oriented approach of dynamic
polymorphism but I would claim that these are implemented at least
with the spirit of STL in mind which is essentially the separation of
algorithms and containers.

> Most of the other libraries are not as comprehensive or as
> coherent as the STL.  It's not unusual to have a vector, a list
> and a map class, all with iterators.  As far as I know, however,
> only STL explicitly categorizes those iterators, and gives you a
> means to determine the category in a template instantiation of
> your own.

Coherence is definitely a trait of STL. Whether I would put
comprehence in there is a different issue: there are important
containers missing (e.g. hashes). However, the coherence enables
the implementation of algorithms which are independent of the
containers because the algorithms don't care at all about the
details of the containers. They merely care about certain concepts
they are built upon to access containers. This is the key difference,
not the presence of containers by itself.

> That doesn't mean that the other libraries are OO --
> I would classify most of them as procedural.  And generic, but
> with a more limited definition of "generic" than that used in
> the STL.

The definitions of the various paradigms are too vague to discuss
this in too much detail, I think. Most of the container libraries
I have seen in C++ use at least encapsulation and object which is
is why I would rather classify them as object oriented. In addition,
the majority also uses dynamic polymorphism in some form or another
which is something unavailable in procedural programming. They are
often "lightweight generic" in the sense that they use templates to
avoid certain casts (much like Java or C# use "generics": just
because they call their static polymorphism "generic" does not mean
that you can do generic programming in either of these languages).

>>>All of these libraries provide alternatives to the STL
>
>> Actually, they don't. They may provide containers as STL does
>> but STL is not about containers.  STL is about algorithms
>> which are entirely independent of containers.  They operate on
>> sequences and containers happen to be examples of these
>> sequences.  I don't think that any of the libraries you
>> mention and most of the libraries you didn't provides an
>> extensive set of algorithms.  Hence, they are no alternative
>> to STL.
>
> IMHO, the algorithms part is open in the STL.

We are definitely in agreement here! Actually, the container part is
also open in the STL. Thinking of it, actually everything is open in
STL: the algorithms, the containers, the functors, even the concepts.

> The algorithms in
> the standard library are good examples of what can be done, and
> some of them are interesting in their own right, but part of the
> point is that I can also furnish my own algorithms.  Algorithms
> which, unlike those designed for another library, will work
> indifferently on vectors, lists, or even istreams.

This is one key! ... or put differently: we are in violent agreement
about STL, I think :-)

> What sets the STL off from other libraries isn't any particular
> bit of code.  It is the unity of concept.  Algorithms and
> iterators aren't two independant concepts -- algorithms are
> conceived in terms of iterators of a specific category, and
> iterators are conceived to conform to a specific category.

Yup.

>> Actually, this "someone" was a committee with all big players
>> in the C++ area being represented and there was obviously a
>> majority to pick this specific library and not others.
>
> Were any others proposed?  Or was it a case of STL or no
> general library?

I wasn't participating in the standardization at this time. My
understanding is that something like NHICL (National Health
Institute Class Library) was under discussion for inclusion but no
formal proposal for its inclusion was received. As far as I
remember discussions about container libraries around this time, I
can very well imagine that everybody felt that there would be no
chance to agree on an interface: there were all kinds of quibbles
about minor details like which of the functions should be virtual
or not. Judging from P.J.Plauger's "The (Draft) Standard C++ Library"
there were only two array-like containers ('dynarray' and
'ptrdynarray') but no list or map going into the standard library
before STL was proposed. ... and I have to admit that I'm still
intrigued by the courage and vision of the committee to accept
something so radically different as the STL into the standard!
-- 
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

wij@seed.net.tw wrote:
> STL but a Template Library among numerous ones, implemented by so
> called 'advanced macro'.

You should have a closer look at STL which is, BTW, kind of a subset
of the standard C++ library. It is quite different from other template
libraries and the classification of templates as "advanced macros" is
also rather inappropriate for C++'s template mechanism.

> So "I am also curious what's the lot part of programming one can miss
> if not using STL? That is interesting.", the contrary might the truth.

STL is based on a programming paradigm which was unknown prior to its
introduction which is now going under the name "generic programming".
I doubt that knowledge of generic programming can be harmful but
consider it a valuable addition to the set of techniques. For many
needs it is the right tool. Also note that it is not really necessary
to use STL at all: you can benefit from the new techniques even
without using STL. Essentially, the techniques are always applicable
when you have some form of algorithm.

> The thing I know about the necessity of STL is not related to C++ but
> stream/formated I/O, locale...

Streams, locale, and probably everything else covered in your "..." are
not part of "STL". STL was originally the name of a library implemented
by HP and today generally refers to the portion of the standard C++
library which is based on this library. It includes essentially the
algorithms, iterators, containers, and functors in the standard C++
library.
-- 
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
>> I consider it an error which is fixed when I detect it. Not being
>> perfect, I may be ignorant about some problems but I never accept
>> deliberate violations like memcopy()ing non-PODs.
>
> Hm, perhaps you never used vector to store anything else than
> fundamental types?

You are not telling me that vector is trying to use memcopy() to
move elements? ... or that the performance of putting classes into
vector is mediocre because the are move using copy construction and
assignment? Neither holds true in my experience: the few cycles
needed to move objects never caused me any problems which I could
not correct with the use of capacity() or the use of a different
data structure (like e.g. a list).

> That said, possibility that my code could run significantly faster by
> makeing one well-thought and safe standard violation just makes
> restless.

What is a "well-though and safe standard violation"? Something which
the standard states it not a violation and thus safe? ... or something
which has just not yet blown into your face because you better don't
port to other systems? My understanding is that the standard guarantees
zip if you violate the rules set out by it. I'd rather not face a trial
because something failed miserably and caused some damage due to a
deliberate violation of the rules.
-- 
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler napisa�(a):
> > [...]
>
> Well, loops based on indicies have usually i < c.GetCount() (or i <
> size() if you insist) condition. That rules out most of problems.

What problems?

> This action might modify the container.

And as you say "maigth" which does not mean they always "do".
I think you have to know what operation(s) will be
peformed in your loops, if they will be mutating or not.
Also, that is the reason you are using const accessor (iterator) or not.

> If this accidentally happens with iterators and
> say vector, you are out of luck (iterators get invalidated, or get out
> of range and you are testing just equality to End).

No! In such case you are a bad programmer!
I don't know how it is possible to write program and not know along what
does your program (does it remove container item or not).


> With indicies it usually works as expected out of box without trouble.

Hm, it does not matter.
As I see you don't understand differences and commons between indices
and iterators.

> And, BTW, performance wise, there is no difference either. Compiler will
> emit similary effective code regardless you are using indicies or iterators.

....but the thread is about using STL algorithms...so here come iterators

>>2.) std::vector<std::string>::iterator is something, for which I flunk
>>pupils.  Over 40 ones.  Is it really necessary to repeate that
>>std::vector<std::string> in 254 places of the code?  How about having a
>>
>>typedef std::vector<std::string> str_table_t;
>>
>>inside that class having that very container as a member?
>
>
> Great. Now you had to introduce another type just to keep your broken
> machine going. Adding entities to code just to make your loops run does
> nothing than increase your code complexity.

*wrong*. It seems you are complaining about whole C++
"that construction isn't beautifull because there are
some strange asterix's and ampersands, I won't use it, feee"
so don't use it.

>>So if tomorrow I
>>decide to use list, or deque
>
>
> So what? How str_table_t is going to help you? Do you plan to replace
> ALL occurences of std::vector<std::string> in your code with list or
> deque?


Yes, and it will work if you take it to consideration at the start
of programming. Jus 15 minutes of thinking (if you know STL containers
well) and then you will be prepared (you won't use incompatibilities, if
you do not have to) to change container that way.

> Anyway, for me this is much more simple. If I want to replace one of my
> vector-like container with another, I just do that

What will take hours and will produce 2 separate projects (source code
repos) with just one difference.

> They have similar
> interface and no additional types needed to write code with them (thanks
> to indicies).

No additional type but many copy-pastes.
What is wrong with additional types?
In OOP you create types in hundreds, every class is new type,
so why not to use one simple and VERY descriptive (for readability of
your code) type?

I use typedefs very frequently and I create type of collection I want to
use. It saves me writting when I want to declare iterators (get rid of
all those scope operators :: and <> braces :-)

typedef std::vector<std::string> students_t:
then I can you simple students_t::const_iterator
and then I know from a first look what does this container
(iterators,subtypes, etc.) operate on...students.

Cheers


-- 
Mateusz �oskot
mateusz at loskot dot net


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
>>What matters is that you see *this one line* in your "main code":
>>
>>count_if (names.begin(), names.end(), longer_than("Carlos"));
>>
>>And that, not only is a single line (vs. four lines, without counting
>>the curly braces, which I personally always spend one line for each
>>of them, so that would make *eight* lines), but it makes it immediately
>>obvious that you're counting how many strings are longer than my name.
>
>
> Well, but only as long as you know what 'longer_than' does. Otherwise it
> will mean studying code at completely unrelated place and remembering
> meaning of another symbol.
>

Exactly.  I think its overkill to try to functionise these one liners.
What code are we re-using here?  A single > expression?

Its replacing a one line test with a once off separate class of a half
dozen lines or so, located who knows where.  Hmm, turns out we want to
count >= instead.. ok, I better make up a new class.

Even this simple example isn't particularly easy to understand.  Is it
counting instances where n.length > "Carlos", or where "Carlos" >
n.length?  Consider the C++0x equivalent, using the new 'for' extension:

int count = 0;
for (string& n : names)
{
    if (n.length() > s.length())
    {
      ++count;
    }
}

Once the new 'for' loop starts being used and recognised, there won't be
much need to make up a whole function object for these trivial cases.
Of course its a different story when we start getting into algorithms
that do real work.

--
Geoff

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"White Wolf" <wolof@freemail.hu> writes:

| Gabriel Dos Reis wrote:
| > "White Wolf" <wolof@freemail.hu> writes:
| >
| >> In what way are indices more likely to stay valid than iterators?  I
| >> mean if I have index #10 tored away for a vector, and then I remove
| >> stuff and have only 5 elements, my index becomes rogue just the same
| >> way an iterator would. I just do not see (or believe) that it is any
| >> less likely with indices than iterators.
| >
| > push_back is most likely to invalid previous iterators than shrinking
| > a vector.
|
| That is why you will use indices with a vector (and not int, but size_t).

Reasonable people have diverging opinions on whether size_t shall be
used as indices for arrays or vectors (yes, I know what the standard
says).  See recurent discussions on the issue.  Personally, I believe
size_t is a distraction.  Do you suffix your integer literals with "u"
or do you cast them to size_t before use as an index into a vector?  I
do not.  And most major C++ textbooks I know of don't.

| But how does that prove that all iterators shall be banished and we should
| chanpe maps to hashes to satisfy this nonsense?

That is an interesting question, but I have not seen people suggest
that all iterators shall be banished.


At the extreme, one can consider an integer value as a canonical form
of iterator -- if you think of a sequence as a mapping from |N to
<value>, then the domain of the mapping is a set of iterators.  Of
course, there are various irregulatiries that do not feet nicely in
that picture, but most well-behaved sequences do.

-- 
                                                        Gabriel Dos Reis
                                            gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> Wu Yongwei wrote:
> > Mirek Fidler wrote:
> >
> >>Anyway, more importantly, I believe that O(1) insertions advantage is a
> >>myth. Before you can apply such insertion, you have to find place where
> >>to insert and list iteration is much slower than vector iteration.
> >>
> >>I am afraid that number of real world problems where list has any
> >>advantage is pretty close to zero. I have not found any real use for
> >>list container since I have started using C++ (for 12 years). (BTW, this
> >>does not apply at list data structure, just as container it is useless).
> >
> >
> > I am really puzzled.  What can be the more natural representation of
> > the list data structure than the list container?
>
> Well, it is just virtually each time I need linked list kind of thing, I
> always need it in non-container form.
>
> E.g. I need to group objects regardless their actuall placement (they
> can by on heap, on stack, in container, whatever). Then double-linked
> list is very natural thing to use, but obviously deque will not do it...
> (because those objects are already owned).

I will try list<Obj*> (or deque<Obj*>).

> > I use std::list a lot, for different purposes.  Very often it is just a
> > container of pointers to maintain some objects' lifetime.  Other
> > containers can do too, but I find list the easiest and the fastest.
>
> Well, "fastest" depends. Have ever done actual measurements?
>
> vector-like containers (esp. NTL Vector) always outperfrom list in
> operations like push_back or iterations.

Standard containers always have an advantage over non-standard ones.
My feeling about NTL is that it is not the C++ way proper.  The use of
`=' for move semantics is confusing, to say the least.

I feel that Andrei Alexandrescu's study in this aspect is more
promising to produce a more `C++' way.

> Of course, insert and erase at arbitrary position is a problem for
> vector, but then again I think that my above statement about finding
> position in list is generally true. I mean, while list insert/erase
> itself is O(1), "where to insert" problem is likely to make total
> performance worse.
>
> Mirek

If I use lists, then I always access them sequentially from the
beginning or the end, or store the iterators of some points that I am
specially interested in in other containers.  I am never worried with
`where to insert' problem.

Best regards,

Yongwei


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Gabriel Dos Reis wrote:
> "Wu Yongwei" <wuyongwei@gmail.com> writes:
>
> | Mirek Fidler wrote:
> | > Anyway, more importantly, I believe that O(1) insertions advantage is a
> | > myth. Before you can apply such insertion, you have to find place where
> | > to insert and list iteration is much slower than vector iteration.
> | >
> | > I am afraid that number of real world problems where list has any
> | > advantage is pretty close to zero. I have not found any real use for
> | > list container since I have started using C++ (for 12 years). (BTW, this
> | > does not apply at list data structure, just as container it is useless).
> |
> | I am really puzzled.  What can be the more natural representation of
> | the list data structure than the list container?
>
> One might argue whether one is considering "list" as synonymous of
> "finite sequence" or "lits" as "linked list" :-)

I think most programmers will of course think of linked lists, singly
linked or doubly linked.

>
> | I use std::list a lot, for different purposes.
>
> That is very interesting.
> Very recently, I was asked the question "do you know of a genuine use
> of (linked) list that is simple to explain?"  Since the question was
> asked by someone who obvisouly is not what is called a beginner, I
> paused a bit before risking my first tentative (which turned out to be
> "wrong" anyway).  Then, we spent a couple of days before reaching some
> kind of agreements on something.  The key observation that killed most
> of the examples was that each time one believes that a list is
> needed/suitable, actually one was thinking of the abstract notion of
> "sequence" or "list", not the "linked list" data structure.  Usually
> the items are part of other bigger data structures and are chained
> together by conceptual links and similar things. And a better and simpler
> representation always seemed to exist.
>
> So, this is to say that I'm highly interested in your frequent uses of
> std::list both for my education and curiosity -- the key part i
> "simple to explain".

I must say that when the I use the std::list container, often I am
*not* using the list data structure.

> | Very often it is just a
> | container of pointers to maintain some objects' lifetime.  Other
> | containers can do too, but I find list the easiest and the fastest.
>
> Indeed, I'm wondering why std::vector would not have been a better
> representation in terms of simplicity and efficiency.

Inserting at the beginning of a vector is O(n), and at the end
amortized O(1).  For lists, they are simply O(1).


Best regards,

Yongwei


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

wij@seed.net.tw wrote:
> Gene Bushuyev wrote:
> >...
> >So if a person doesn't understand most part of STL then he probably missed
> >a hole lot of C++. Just a thought.
>
> I have no opinion how one should or should not like/use STL.
> But I am also curious what's the lot part of programming one
> can miss if not using STL? That is interesting.

As another poster says, you can always use `while' instead of `for'.
You lose the elegance and compactness of expressions.

Using STL is really an easier way for beginners to express abstract
data structures than handmade structures.  Compare:

--- Using STL
class Obj { ... };
typedef list<Obj> ObjList;

--- Not using STL
class Obj { ... };
struct ObjList {
   Obj m_obj;
   Obj* m_prev;
   Obj* m_next;
};

Oftener than not people in the latter case will mingle the object
pointers into the class definition itself owing to the habits from C.
It simply makes a mess between model and representation.

Another example is the sort algorithm.  Sort executes faster than
qsort, and probably your own handmade sort implementation.

Best regards,

Yongwei


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Francis Glassborow wrote:
> In article <m364utrimr.fsf@uniton.integrable-solutions.net>, Gabriel Dos
> Reis <gdr@integrable-solutions.net> writes
> >Indeed, I'm wondering why std::vector would not have been a better
> >representation in terms of simplicity and efficiency.
>
> One of the interesting observations mad several years ago by Andy Koenig
> was that whilst std::list seemed to have the edge over std::vector
> because of the 'ease' with which items could be inserted and deleted
> that may be based on theory rather than practice.
>
> He then went on to point out that with modern multi-level cache systems,
> the contiguous nature of the vector's data storage can result in a major
> performance gain, even an insert may be faster for small to medium size
> data structures.

Only true if insertions (and erasures) occur at the end of the
container.

> That leaves me wondering whether decisions to use std::list are not
> often cases of premature optimisation.

Even if insertions and erasures occur at the end of vector, std::list
will ensure predictability, an often-needed feature that is only
exhibited by languages like C and C++.


Best regards,

Yongwei


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> 2.) I still do not see *any* reason to change a map to a hash, just because
> I am afraid of iterators.

Well, let me try. What about speed?

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Dietmar Kuehl wrote:
> Mirek Fidler wrote:
>
>
>>Well, I am addressing issue when some elements from container are
>>removed while performing loop, compare
>>
>>for(std::vector<foo>::iterator it = bar.begin(); it != bar.end(); ++it)
>>     if(condition(*it))
>>        bar.erase(it);
>>
>>for(int i = 0; i < bar.GetCount(); i++)
>>     if(condition(bar[i]))
>>        bar.Remove(i);
>
>
> Both loops are a stupid idea in the first place since the will miss

Well, my mistake, I should have invented something real to illustrate
the problem. But I have to agree that this issues is much less important.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> And as you say "maigth" which does not mean they always "do".

That make it even worse, does not it?

> I think you have to know what operation(s) will be
> peformed in your loops, if they will be mutating or not.

Sure, if you are ideal being, you write just correct software which 
out-of box does what customer demands.

I am not of such a kind. Quite often I have to rewrite loop. My way 
gives me less oportunities to blow things up.

> ...but the thread is about using STL algorithms...so here come iterators

Thread is about whether it is worth to implement trivial loops using 
algorithms.

I just suggested that there is nothing wrong with implementing trivial 
loops using iteration with indicies. It has some advantages.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> If I use lists, then I always access them sequentially from the
> beginning or the end, or store the iterators of some points that I am

Just to showcase my point:

#include <Core/Core.h>
#include <stdio.h>
#include <iostream>
#include <fstream>
#include <time.h>
#include <vector>
#include <algorithm>
#include <map>
#include <deque>
#include <string>
#include <list>

using namespace std;

const int REP = 100000;
const int N   = 1000;
const int AND = 31;
const int LIM = 15;

inline bool llim(int x)
{
	return x < LIM;
}

CONSOLE_APP_MAIN
{

	Vector<int> seq;
	for(int i = 0; i < N; i++)
		seq.Add(rand() & AND);
	for(int n = 0; n < REP; n++) {
		{
			Vector<int> s(seq, 1);
			RTIMING("Vector trivial");
			Vector<int> t;
			for(int i = 0; i < s.GetCount(); i++)
				if(s[i] >= LIM)
					t.Add(s[i]);
		}
		{
			Vector<int> s(seq, 1);
			RTIMING("Vector reserve");
			Vector<int> t;
			t.Reserve(s.GetCount());
			for(int i = 0; i < s.GetCount(); i++)
				if(s[i] >= LIM)
					t.Add(s[i]);
		}
		{
			list<int> l(seq.Begin(), seq.End());
			RTIMING("list remove_if");
			l.remove_if(llim);
		}
	}
}

TIMING list remove_if :   1.2 s
TIMING Vector reserve : 696.8 ms
TIMING Vector trivial : 747.8 ms

For LIM = 1 (low count of elements removed)

TIMING list remove_if : 402.3 ms
TIMING Vector reserve : 382.3 ms
TIMING Vector trivial : 438.3 ms

For LIM = 30 (most elements removed)

TIMING list remove_if :   1.5 s
TIMING Vector reserve : 386.0 ms
TIMING Vector trivial : 311.0 ms

(Sorry for using U++/NTL; RTIMING measures time to the end of scope...)

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> You are not telling me that vector is trying to use memcopy() to
> move elements? ... or that the performance of putting classes into
> vector is mediocre because the are move using copy construction and
> assignment? Neither holds true in my experience: the few cycles
> needed to move objects never caused me any problems which I could
> not correct with the use of capacity() or the use of a different
> data structure (like e.g. a list).

Real world example: Simple text editor, lines stored as std::vector< 
std::string >. This is the most natural way, list has no advantage here 
(you need to have random based access to e.g. paint your lines on the 
screen).

Now to insert lines, you have to perform insert. Using my violation, 
this will be 15 times faster. Means that my text editor will be able to 
reasonable work with files 15 times longer than yours (or you will be 
forced to find more complicated solution).

For me it this is worth it.

>>That said, possibility that my code could run significantly faster by
>>makeing one well-thought and safe standard violation just makes
>>restless.
> 
> 
> What is a "well-though and safe standard violation"? Something which
> the standard states it not a violation and thus safe?

Something that is nearly impossible to break and when it breaks, it is 
nearly impossible not to notice.

I am not THAT good in logic, but I believe that current standard in fact 
requires some behaviour that is in accordance with my violation. In 
fact, all that I require that compiler does not change layout of object 
when constructor/non-virtual destructor is added. Given that inheritance 
has to work, this one is hard to break.

Think: you have type T which is POD and T1 which is inherited from T and 
has just adds constructor and/or destructor. Now if you are supposed to 
access T members through pointer to T1 converted to T, it is obvious 
that T has same layout as T1. Well, pehaps some sinister compiler writer 
could imagine a way how to spoil this effect, but I guess it is unlikely 
and maybe even impossible - I believe that something else in standard 
compliance of such compiler would break.

And above is all that I require.

> because something failed miserably and caused some damage due to a
> deliberate violation of the rules.

There is milion of things that can fail, and 999 999 is more likely to 
fail than this one.

In the end, if this is about to blow, it will blow immediately. 
(Actually, you can check it easily at application startup. You do test 
your applications before shipping them, do you?)

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <1122791044.482878.125080@o13g2000cwo.googlegroups.com>, Wu 
Yongwei <wuyongwei@gmail.com> writes
>> He then went on to point out that with modern multi-level cache systems,
>> the contiguous nature of the vector's data storage can result in a major
>> performance gain, even an insert may be faster for small to medium size
>> data structures.
>
>Only true if insertions (and erasures) occur at the end of the
>container.

I think you miss the point. On a system where all memory accesses take 
the same time the theoretical analysis for time to insert favours a list 
as long as you do not have to traverse the list to find the insertion 
point (which is actually quite a big if). However modern systems have at 
least three levels of storage (cache, main memory and backing store -- 
used as virtual memory) and many have five or more levels (with three 
levels of cache).

If the whole of a vector fits in the first level data cache it is quite 
likely that it outperforms a list where the nodes are scattered.

>
>> That leaves me wondering whether decisions to use std::list are not
>> often cases of premature optimisation.
>
>Even if insertions and erasures occur at the end of vector, std::list
>will ensure predictability, an often-needed feature that is only
>exhibited by languages like C and C++.

Predictability? I assume you are referring to objects remaining at the 
same address. That is also true of a deque.

I will repeat, choice of sequence container is often an optimisation. 
Experience teaches us that optimisation is not something that we get 
intuitively right, not least because our decisions are too often based 
on a false view of the way a system works in practice.

We should also note, that the way systems worked 10 years ago is a poor 
guide to the way they work today, or will work in 10 years time.


-- 
Francis Glassborow      ACCU
Author of 'You Can Do It!' see http://www.spellen.org/youcandoit
For project ideas and contributions: http://www.spellen.org/youcandoit/projects


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> Inserting at the beginning of a vector is O(n), and at the end
> amortized O(1).  For lists, they are simply O(1).

There is deque.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

>>E.g. I need to group objects regardless their actuall placement (they
>>can by on heap, on stack, in container, whatever). Then double-linked
>>list is very natural thing to use, but obviously deque will not do it...
>>(because those objects are already owned).
> 
> 
> I will try list<Obj*> (or deque<Obj*>).

Yes, but that would mean storing back-iterator in Obj. That is way too 
complicated and error-prone.

I have rather created simple helper class called "Link" that adds all 
linking operations to Obj this way

struct Obj : Link<Obj> {
}

and now Obj has Link, Unlink etc...

I believe that this is better abstraction for this.

>>vector-like containers (esp. NTL Vector) always outperfrom list in
>>operations like push_back or iterations.
> 
> 
> Standard containers always have an advantage over non-standard ones.
> My feeling about NTL is that it is not the C++ way proper.  The use of
> `=' for move semantics is confusing, to say the least.

Yes. I spend years searching for better alternative.

Anyway, it is confusing for first 5 minutes. In reality, only containers 
itself are implemented this way (and then things that contain containers 
as members). In practice, it is very easy to stay

> I feel that Andrei Alexandrescu's study in this aspect is more
> promising to produce a more `C++' way.

MOJO? Well, that is great idea, only problem is that it does not scale 
as well. When composing, I get my "pick" for free, and that is usually 
exactly what I want to have (if not, I can always use WithDeepCopy).

> If I use lists, then I always access them sequentially from the
> beginning or the end,

Like adding & removing elements when iterating them? Using two Vectors 
could be faster.

> or store the iterators of some points that I am
> specially interested in in other containers.  I am never worried with
> `where to insert' problem.

Once again, while in theory this is the only apology for list existence, 
I never met real world situation where "store the iterators of some 
points" part applies (apart from some sort of backpointers as shown 
above, but that is hardly optimal solution).

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> Even if insertions and erasures occur at the end of vector, std::list
> will ensure predictability, an often-needed feature that is only
> exhibited by languages like C and C++.

Perhaps on CP/M machine. There is zillion of things that can happen and 
completely spoil predictability. Cache line misses, allocator issues, 
pipeline stalls, page swaps...

Performance today is stochastic thing. Never expect, always measure.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> 		{
> 			Vector<int> s(seq, 1);
> 			RTIMING("Vector reserve");
> 			Vector<int> t;
> 			t.Reserve(s.GetCount());
> 			for(int i = 0; i < s.GetCount(); i++)
> 				if(s[i] >= LIM)
> 					t.Add(s[i]);
> 		}

Stupid me, I missed the most effective solution (ok, 1 point in favor of 
algorithms :):

		{
			Vector<int> s(seq, 1);
			RTIMING("Vector single");
			int q = 0;
			for(int i = 0; i < s.GetCount(); i++)
				if(s[i] >= LIM)
					s[q++] = s[i];
			s.SetCount(q);
		}

TIMING list remove_if :   1.3 s
TIMING Vector single  : 527.6 ms
TIMING Vector reserve : 644.6 ms
TIMING Vector trivial : 767.6 ms

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
>> 2.) I still do not see *any* reason to change a map to a hash, just
>> because I am afraid of iterators.
>
> Well, let me try. What about speed?

What about it?  What speed?  Speed of programming?  I have a class with 101 
members in it (it is called NineteenEightyFour), and it has a less-than 
operator.  I do *no* lookup, I need them ordered and stay ordered.  How long 
does it take for me to put it into a map?  5 seconds.  How long does it take 
for me to write a proper* has function for it?  Bloody ages.  Does a hash 
give me ordered iteration?  Nope.

So let's go, and complitace my code beyond belief, so that I can use an 
index to iterate in an order I never wanted. :-(((

* Proper speed, proper distribution etc.

-- 
WW aka Attila
:::
There is always one more imbecile than you counted on. 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Gabriel Dos Reis wrote:
> "White Wolf" <wolof@freemail.hu> writes:
>
>> Gabriel Dos Reis wrote:
>>> "White Wolf" <wolof@freemail.hu> writes:
>>>
>>>> In what way are indices more likely to stay valid than iterators?  I
>>>> mean if I have index #10 tored away for a vector, and then I remove
>>>> stuff and have only 5 elements, my index becomes rogue just the same
>>>> way an iterator would. I just do not see (or believe) that it is any
>>>> less likely with indices than iterators.
>>>
>>> push_back is most likely to invalid previous iterators than shrinking
>>> a vector.
>>
>> That is why you will use indices with a vector (and not int, but
>> size_t).
>
> Reasonable people have diverging opinions on whether size_t shall be
> used as indices for arrays or vectors (yes, I know what the standard
> says).  See recurent discussions on the issue.  Personally, I believe
> size_t is a distraction.  Do you suffix your integer literals with "u"
> or do you cast them to size_t before use as an index into a vector?  I
> do not.  And most major C++ textbooks I know of don't.

I do.  Which reminds me of what all the billions of flies do... and it is 
still not a good thing.

>> But how does that prove that all iterators shall be banished and we
>> should chanpe maps to hashes to satisfy this nonsense?
>
> That is an interesting question, but I have not seen people suggest
> that all iterators shall be banished.

I did.

> At the extreme, one can consider an integer value as a canonical form
> of iterator -- if you think of a sequence as a mapping from |N to
> <value>, then the domain of the mapping is a set of iterators.  Of
> course, there are various irregulatiries that do not feet nicely in
> that picture, but most well-behaved sequences do.

It is (IMHO) a simplification, which has no reason to exist, other than old 
habbits.

-- 
WW aka Attila
:::
Accidents don't just happen. They must be carelessly planned. 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
>> And as you say "maigth" which does not mean they always "do".
>
> That make it even worse, does not it?

Nope.  Since if a programmer does not *know* when this could happen, that 
programmers must not be let close to any program.

>> I think you have to know what operation(s) will be
>> peformed in your loops, if they will be mutating or not.
>
> Sure, if you are ideal being, you write just correct software which
> out-of box does what customer demands.

???  What is the sarcasm for?  You do not know if your loop is mutating or 
not?  I mean man!  How do you write code if you haven't got the faintest 
clue what it does?

> I am not of such a kind. Quite often I have to rewrite loop. My way
> gives me less oportunities to blow things up.

You mean you rewrite a pure reading iteration into one which writes?  I have 
hard time to imagine *that* *bad* design where this can happen.  And believe 
me, I did see a lot of most awful designes.

>> ...but the thread is about using STL algorithms...so here come iterators
>
> Thread is about whether it is worth to implement trivial loops using
> algorithms.

Nope, it is not:

"It boils down to asking whether it's really reasonable to expect C++ 
programmers to use STL algorithms instead of writing their own loops"

There is no word *trivial* in the whole OP.  There is however nontricial... 
which suggests just the very opposite to what you state.

> I just suggested that there is nothing wrong with implementing trivial
> loops using iteration with indicies. It has some advantages.

Surely it does.  It is familar.  And that is about all the advantages it 
has.  And in some corner cases (which may happen in 0.001% of the code we 
write) it may make the code work (referring to reallocation) and in 100% of 
such code it may hide errors and undefined/undesigned loop invariant 
(referring to the < instead of !=).

-- 
WW aka Attila
:::
It might look like I'm doing nothing, but at the cellular level I'm really
quite busy. 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Dietmar Kuehl wrote:
>..
>Streams, locale, and probably everything else covered in your "..." are
>not part of "STL". STL was originally the name of a library implemented
>by HP and today generally refers to the portion of the standard C++
>library which is based on this library. It includes essentially the
>algorithms, iterators, containers, and functors in the standard C++
>library.

Probably I had over-stated in the previous responding.

It reminded me of an early version of HP template library, and ideals
of that time. I have been hoping some real result(relative to me) in
the past. But continously seeing new break-through technology turned
out te be another old bottle. BTW, Depending on which aspect you
emphyasize, "programing in advanced macro" or "generic programming",
both looked ok for me. For instance, the one I cared much is vector
because of its 'macro' nature which makes me using it mostly in trival
programs.

One of Meyers's questions: whether it is practical to replace many
manual loops with algorithm calls. I guess the answer will be no.
People seemingly just go on programming their way (out of their best
and free choice, of course).


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

White Wolf wrote:
> Mirek Fidler wrote:
> 
>>>2.) I still do not see *any* reason to change a map to a hash, just
>>>because I am afraid of iterators.
>>
>>Well, let me try. What about speed?
> 
> 
> What about it?  What speed?  Speed of programming?  I have a class with 101 
> members in it (it is called NineteenEightyFour), and it has a less-than 
> operator.  I do *no* lookup, I need them ordered and stay ordered.  How long 
> does it take for me to put it into a map?  5 seconds.  How long does it take 
> for me to write a proper* has function for it?


If your class has 101 members,

a) how long it takes to you to write proper ordering predicate if you 
have operator< available for all members
b) like a), but you have operator== too.

> Bloody ages.  Does a hash 
> give me ordered iteration?  Nope.

Does map give me stable order? Nope. Sorting is easy and relatively 
cheap. Lost input order is hard to maintain. (By stable order I mean 
that elements can be iterated in order they entered associative container).

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> > You are not telling me that vector is trying to use memcopy() to
> > move elements? ... or that the performance of putting classes into
> > vector is mediocre because the are move using copy construction and
> > assignment? Neither holds true in my experience: the few cycles
> > needed to move objects never caused me any problems which I could
> > not correct with the use of capacity() or the use of a different
> > data structure (like e.g. a list).
>
> Real world example: Simple text editor, lines stored as std::vector<
> std::string >. This is the most natural way, list has no advantage here
> (you need to have random based access to e.g. paint your lines on the
> screen).

Are you saying you have written a text editor this way? Because this
sounds like a pretty poor approach to me, and not for the reasons you
state. And yes, std::list<std::string> would be a much better approach
(you need efficient insertion far more than you need random access
iterators; for display purposes you just need an iterator to the first
displayed line, whether it's random access or not is irrelevant), but
still a pretty bad idea. std::deque<std::string> would probably be
better, but for best results I wouldn't use std::string at all. "A text
document is a collection of text lines" is simply the wrong
abstraction.

> >>That said, possibility that my code could run significantly faster by
> >>makeing one well-thought and safe standard violation just makes
> >>restless.
> >
> >
> > What is a "well-though and safe standard violation"? Something which
> > the standard states it not a violation and thus safe?
>
> Something that is nearly impossible to break and when it breaks, it is
> nearly impossible not to notice.
>
> I am not THAT good in logic, but I believe that current standard in fact
> requires some behaviour that is in accordance with my violation.

Nope; the standard doesn't reqiuire any behavior when you break the
rules. That's why it's called "undefined behavior". It even allows the
system to "work", which is why you can get away with it in your
environment.

> In
> fact, all that I require that compiler does not change layout of object
> when constructor/non-virtual destructor is added. Given that inheritance
> has to work, this one is hard to break.
>
> Think: you have type T which is POD and T1 which is inherited from T and
> has just adds constructor and/or destructor. Now if you are supposed to
> access T members through pointer to T1 converted to T, it is obvious
> that T has same layout as T1.

Again, nope; what the standard says is that if you have a pointer to a
T1 and convert it to a pointer to a T, the conversion works. It doesn't
say anything about T1 being laid out the same way as the T. For
example, a T1 could have data before any of the T members, which would
be the case with:

struct B {
   int x;
};

class T1 : public B, public T {
};

> > because something failed miserably and caused some damage due to a
> > deliberate violation of the rules.
>
> There is milion of things that can fail, and 999 999 is more likely to
> fail than this one.
>
> In the end, if this is about to blow, it will blow immediately.

How do you know that? And even if it does, what will you do about it?

> (Actually, you can check it easily at application startup. You do test
> your applications before shipping them, do you?)

Like Dietmar, I follow a "standard violations are unacceptable" policy.
I can't speak for his reasons, but in my case it's because I've been
bitten several times in the past when trying to get away with "one
little violation, for which I have really good reasons." What always
happened was that eventually the violation blew up when the system was
scaled up. Sometimes the scaling was due to uses I didn't anticipate
(like designing a system with single iheritance in mind, then using
multiple inheritance). Sometimes it was due to porting it to a new
OS/compiler combination (sometimes even just a new version of my
current compiler was enough). In every case, the standard violating
code/design had to be backed out and redone in a standard-conforming
way.

Eventually I realized it was better to just stay within the standard in
the first place; opening the door on standard violations, while giving
the appearance of working today, is just going to hurt later when the
system grows beyond the environment in which it appears to work. Maybe
it won't happen to you. Maybe you won't have to grow your system in a
way that makes the problem intractable. Maybe you're just smarter than
me. But I find it simpler to just stay within the boundaries of
standard C++. It gives me one less thing to worry about.

Bob


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler napisa�(a):
>>I think you have to know what operation(s) will be
>>peformed in your loops, if they will be mutating or not.
> 
> 
> Sure, if you are ideal being, you write just correct software which 
> out-of box does what customer demands.

No I'm not ideal at all but I usualy try to know what my program does.

> I am not of such a kind. Quite often I have to rewrite loop. My way 
> gives me less oportunities to blow things up.

The problem you are pointing about mutate or not container elements
is a kind of simple and beginner's problem.
In this case loops can be obviously catetorized to 2 categories:
- reading from container
- writtng (may be reading in the same loop) or mutating container elements

IMHO it should be very obvious problem.

> I just suggested that there is nothing wrong with implementing trivial 
> loops using iteration with indicies. It has some advantages.

Yes, there is nothing wrong if you implement such loops propely and
wich performance related hints in mind.

I'd like to say implementing loops using iterators has
more advantages (see: algorithms) than using indexing.

Cheers
-- 
Mateusz �oskot
mateusz at loskot dot net


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Marco Manfredini napisa�(a):
> Scott Meyers wrote:
> 
>>[...] The other is where people can find open-source examples of real
>>systems that actually do it.
> 
> 
> For that question: As a nonrepresentive estimate, I've put together a
> script that scans all the c++ source files (ie: C|cpp|cc|H|cpp|hh) from
> the slackware 10.1 source cd's for the pattern: (find_if|remove_if
> for_each|bind1st|bind2nd). 
> [...]
> 
> The total number of c++ source lines scanned was: 8811927

Great gadget, thanks.

> Not sure if that means something...
> 

Hm, I would say that mean programmers getting interested and learn new 
features of STL, and they try to use them.

I think one of reasons STL algorithms are not very often is that
many programmers do not try to learn and use newest features.
As C++ is a successor of C (and not only C) and C++ programmers
were programming in C before, and they moved to C++, so they
stick, consciously or not, to C paradigm in C++.
That is a kind of barier  before going further and learn more specific 
C++ features.

I think similar, but not so bad, situaton is with templates usage.
I've not done any statistics, but I know many C++ programmers, even in 
my work and templates are used as comming fromexternal libraries like
Microsoft ATL or WTL, but rarely created by programmers.

I believe this situation will change and new features will be used
commonly. I think such authors as Scott Mayers, Herb Sutter and
Nicolai Josuttis do great contribution in this field.

Cheers

-- 
Mateusz �oskot
mateusz at loskot dot net



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Rodrigo Strauss napisa�(a):
> With the .NET marketing avalanche, I
> took some time do think how I would reengineer the software in C#. My
> conclusion was that I would be less productive with the
> "high-productivity languages" (C#, Java) than I am using C++ and STL.

I have similar conclusion.

> I see no reason to use loops when you have a STL algorithm, everyone
> can figure out what std::for_each does, even a non C++ programmer. 

IMHO reading for_each and functor used by is very simple.
Reader don't need to analize how iterationi incluence each other
and he can take a look only at opeartor()(std::string name) {...} of the 
functor and read it like "read "name" and send write it to column X in 
table Y in database Z". I think it is little simplification.


> I use maps, vectors and STL algorithms to control and filter connected
> users and the assets users want to view/sign. The STL code is so
> trivial, so verbose, that I use it to show people that C++ with STL is
> not as hard as most people think.
> 
> Example:
> 
> for_each(clients.begin(), clients.end(), CancelAllSignatures());
> 
> I think it's much better than a loop.
> [...]

That's what I'm talking about :-)

> 
> And I sure agree we need better error messages from the compiler when
> using templates.

Certainly, agreed.

Cheers

-- 
Mateusz �oskot
mateusz at loskot dot net


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> Are you saying you have written a text editor this way? Because this
> sounds like a pretty poor approach to me, and not for the reasons you
> state. And yes, std::list<std::string> would be a much better approach
> (you need efficient insertion far more than you need random access
> iterators; for display purposes you just need an iterator to the first
> displayed line, whether it's random access or not is irrelevant),

Have you ever implemented text editor in X11 or Win32? Or anything else? 
Be honest.

I guess your are just making bold statements. list implementation is 
much more complex and certainly much slower.

In fact, GUI development is always based on indicies. You convert screen 
  coordinates to lines. You get scrollbar positions as numbers. You send 
numbers as position... Only thing you could actually gain by iterators 
and list here is that you will be constantly iterating through list and 
counting screen positions. Very stupid idea.

> but
> still a pretty bad idea. std::deque<std::string> would probably be

OMG, why? deque is totaly wrong type of container here. What you plan to 
gain using deque? Fast insertion of first line?!

>>There is milion of things that can fail, and 999 999 is more likely to
>>fail than this one.
>>
>>In the end, if this is about to blow, it will blow immediately.
> 
> 
> How do you know that? And even if it does, what will you do about it?

Well, maybe I know more than you?

> bitten several times in the past when trying to get away with "one
> little violation, for which I have really good reasons." What always
> happened was that eventually the violation blew up when the system was
> scaled up. Sometimes the scaling was due to uses I didn't anticipate
> (like designing a system with single iheritance in mind, then using
> multiple inheritance).

Done.

  Sometimes it was due to porting it to a new
> OS/compiler combination (sometimes even just a new version of my
> current compiler was enough).

Done.

> In every case, the standard violating
> code/design had to be backed out and redone in a standard-conforming
> way.

Well, if some compiler writer will gone crazy, I will simply switch off 
this optimization. That is 20 minutes work.

But I know that this simply will not happen. There is no reason. No 
possible gain.

> it won't happen to you. Maybe you won't have to grow your system in a
> way that makes the problem intractable. Maybe you're just smarter than
> me.

Perhaps just more experienced.

> But I find it simpler to just stay within the boundaries of
> standard C++. It gives me one less thing to worry about.

I do too. With this one single exception. I do not like to break rules, 
absolutely not. But gains are way too high to ignore here.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> White Wolf wrote:
>> Mirek Fidler wrote:
>>
>>>> 2.) I still do not see *any* reason to change a map to a hash, just
>>>> because I am afraid of iterators.
>>>
>>> Well, let me try. What about speed?
>>
>>
>> What about it?  What speed?  Speed of programming?  I have a class with
>> 101 members in it (it is called NineteenEightyFour), and it has a
>> less-than operator.  I do *no* lookup, I need them ordered and stay
>> ordered.  How long does it take for me to put it into a map?  5
>> seconds.  How long does it take for me to write a proper* has function
>> for it?
>
>
> If your class has 101 members,
>
> a) how long it takes to you to write proper ordering predicate if you
> have operator< available for all members

I would say approximately half an hour, if I do it very slowly.  Compared to 
the several days (or perhaphs weeks) it may take to design and write a 
proper hash function.  It is not as simple as comparison you know.  Hashing 
needs to take the input into consideration to create proper distribution. 
And input changes.  So with hashing you add absolutely unneded complexity if 
all you need is ordering.

> b) like a), but you have operator== too.

??? Why would I?  Nothing whatsoever requires to have operator== for using a 
class with map.

>> Bloody ages.  Does a hash
>> give me ordered iteration?  Nope.
>
> Does map give me stable order?

Yes.

> Nope. Sorting is easy and relatively
> cheap.

Relative to a BMW perhaps.

> Lost input order is hard to maintain.

???  Maps do not loose their order, unless of course one designs as you do: 
leaking out implementation details from all the classes.  Even CA-Clipper 
programmers(*) back in the previous millenia knew that updating a *key* 
needs *very* special caring to ensure that all indices get updated.  So the 
rule of thumb was *not* to update them.

(*) At least those, who could hold a job for more than 3 months in shop 
which wasn't owned by their family.

I am getting the feeling that you are trying to solve your insecurities 
about programming in C++ the wrong way.  What you have as a problem is not 
with the language.  It is with your inadequate as well as bad experiences, 
and the fact that you seem to lack the knowledge of essential idioms. 
Essential idioms or I could rather call them patterns, as they aren't unique 
to C++.  And IMHO trying to blame iterators for that might be a good excuse 
and a fond belief, but it only helps with the psychic part of getting over 
the issue.  But not with the technical part, which is equally important.

> (By stable order I mean that elements can be iterated in
> order they entered associative container).

Excuse me???  Why would I want to do that, when I want to create an 
*ordered* container???  Map is for *ordering* *differently* than the order 
you have entered your elements.  That is the whole point of it.

-- 
WW aka Attila
:::
A person whose heart is not content is like a snake which tries to swallow
an elephant (Chinese proverb) 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Dietmar Kuehl <dietmar_kuehl@yahoo.com> writes:

| STL is based on a programming paradigm which was unknown prior to its
| introduction which is now going under the name "generic programming".

I believe that is an extreme statement given

   http://www.stepanovepapers.com/

existing practice in programming languages like Lisp and ML variants,
and googling about "initial algebras" and "algorithmics." 

-- 
                                                       Gabriel Dos Reis 
                                           gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"White Wolf" <wolof@freemail.hu> writes:

| Gabriel Dos Reis wrote:
| > "White Wolf" <wolof@freemail.hu> writes:
| >
| >> Gabriel Dos Reis wrote:
| >>> "White Wolf" <wolof@freemail.hu> writes:
| >>>
| >>>> In what way are indices more likely to stay valid than iterators?  I
| >>>> mean if I have index #10 tored away for a vector, and then I remove
| >>>> stuff and have only 5 elements, my index becomes rogue just the same
| >>>> way an iterator would. I just do not see (or believe) that it is any
| >>>> less likely with indices than iterators.
| >>>
| >>> push_back is most likely to invalid previous iterators than shrinking
| >>> a vector.
| >>
| >> That is why you will use indices with a vector (and not int, but
| >> size_t).
| >
| > Reasonable people have diverging opinions on whether size_t shall be
| > used as indices for arrays or vectors (yes, I know what the standard
| > says).  See recurent discussions on the issue.  Personally, I believe
| > size_t is a distraction.  Do you suffix your integer literals with "u"
| > or do you cast them to size_t before use as an index into a vector?  I
| > do not.  And most major C++ textbooks I know of don't.
| 
| I do.  Which reminds me of what all the billions of flies do... and it is 
| still not a good thing.

I guess that makes you special.

| >> But how does that prove that all iterators shall be banished and we
| >> should chanpe maps to hashes to satisfy this nonsense?
| >
| > That is an interesting question, but I have not seen people suggest
| > that all iterators shall be banished.
| 
| I did.

Please do enlighten me.

| > At the extreme, one can consider an integer value as a canonical form
| > of iterator -- if you think of a sequence as a mapping from |N to
| > <value>, then the domain of the mapping is a set of iterators.  Of
| > course, there are various irregulatiries that do not feet nicely in
| > that picture, but most well-behaved sequences do.
| 
| It is (IMHO) a simplification, which has no reason to exist, other than old 
| habbits.

"old habbits"?  Please explain.

After I sent my message, I realized -- while looking for something
else -- that the above view is already stated in Alexander Stepanov's works 

   http://www.stepanovpapers.com/

Please, explain me why they are "old habbits", what the "new habbits"
are, what are the essential differences and the advantages of the "new
habbits" over the "old habbits".


-- 
                                                       Gabriel Dos Reis 
                                           gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Scott Meyers wrote:
[SNIP]
> From time to time, I need a reality check.  This is one such time.

Hm.  I should learn that from you. :-)

> I got the following email query from a reader.  It boils down to asking
> whether it's really reasonable to expect C++ programmers to use STL
> algorithms instead of writing their own loops, a topic I address in my
> "Effective STL," where I argue that it often is.

If you read the part I have got involved in your thread you will see, that 
it *is*.  Advocates of hand-written loops have posted about 4 loops, *all* 
of them having one or more *serious* bugs in them.  *Every* programmer I 
have worked with has his *loop* bug.  Mine is to forget to "skip" to the 
next element.  And then wonder, what the heck can take more than 20 seconds 
in looping over my 10 test elements...

My opinion is that loops are *not* taught well in schools.  Not C++ loops. 
Loops at all.  So since many people have no clue about loops, they think 
they know them and that they are easy.  Just do a little test.  Ask around 
practicing programmers to tell you the definition of loop-invariant!  You 
will be surprised how many have no clue at all, and how many have *only* a 
blurred idea.  How do you design something you don't know?

This is not bashing other programmers.  I had no idea that loop invariants 
exist until I have read about them in Accelerated C++.  I was programming 
for nearly 20 years already at that time!  And successfully so, and not 
becaused I have coned my way in.  And I would still have a hard time to come 
up with a clear-cut definition of loop-invariant.

So many people have no clear picture of what a loop is.  I am not talking 
about for and while and syntax and semantics.  I am talking about patterns 
of use, or sane subset, or idiomatic usage.  So they make mistakes.  Then 
they make up excuses, like the one that iterators are evil.

Like the debate in this thread talking about vector iterators being 
invalidated by a puch_back during a reallocation.  Oh my.  (Gabi's idea of 
the iterator holding a ref to the vector and an index solves this issue, and 
I guess originally it has been discarded as an idea based on the (proved 
evil) "efficiency" claims.)  I cannot imagine an algorithm, which would need 
to do that, but let's assume there is one.  So we either use deque there 
*or* any other container than the prematurely overoptimized vector and 
string.  Or we use indices or we use Gabi's special iterator (if we insist 
making a generic algorithm, which may not be such a bad idea anyway).

People understand the "coolness" of loops, but not the loops themselves. 
Hey!  I can break_; and continue;, and push_back and...  It's like trying to 
count sheep while the door to the field is open and the sheep are moving 
around.  Hopeless.

> But the reader puts the question so much better.
> It's long, but very much worth reading, IMO:
>
>  In fact, the features I tried to implement with algorithms were
>  different enough from *any* of the examples I found, that I finally
>  just gave up and went back to manual looping.

Experience.  This (especially with the lacking proper template support of 
most compilers up to few years ago) is a new field for most of us.  I did 
just what you have done in many cases, just because I have found that in 
that particular case, and algorithm would not do any better than a loop. 
But it would take a lot more time to make it.  Not because it should, but 
because I am a dummy and I need time to get used to it.  However my excuse 
is quite good: I am only working with very simple things for now, and my 
containers all support indices.  (Arrays and strings behind a class 
template.)  And the actions I need to do do not fit well the bill of an 
algorithm, perhaps because my design lacks consistency as well.  (I would 
have hard time to implement my "garbage collection" routine in my so-called 
string-pack with an alrogithm.  But IMHO that is my designs fault, as it was 
growing pretty fast, it is not optimal.)

>  Now, I did not give up easily (as I generally don't).

Nope, you didn't.  You have made another mistake.  You have tried to learn 
riding a motorbike on the freeway. :-)  I am learning algorithm usage (after 
burning myself just like you did) in my pet projects (if I ever have time to 
do them).  One needs to get used to the new coding style and the new tricks. 
We, who program for over a decade, are prone to forget how tedious it was to 
learn all these things in the first place.  How many days of trial and 
error, how many sketches, how much thinking it took until we have got the 
mental picture reasonably right.  Now, with the algorithms and the binders 
and lambdas and the functors and all, we have a whole new ballgame to learn 
again.  We just have to remember that while we did have learnt a lot with 
experience, it still does take time and practice to change our design/coding 
paradigms.

It is (I guess) like with metaprogramming.  When it cliks in, it is all very 
simple.  For me, this has happened (thanks to the new metaprogramming book, 
and Dave Abrahams' excellent questions and strict guidance when I have 
mailed him in my deepest frustration) just few months ago.  Now I make 
complex new classes just by parameterizing my class-generator.  It is like 
magic.  But it took me - basically - 4 years to get there!  And while I am 
not a particularly bright guy (ask anyone who knows me ;-) ) I am not stupid 
(don't ask about that one, opinions differ ;-) ).  But it did take me that 
long, because I had a lot to understand about templates and all, name lookup 
etc.

>  I put days (weeks?) into trying to figure this out.

One cannot learn new tricks under the pressure of commercial software 
development.  The pace is so high, that it is not possible.  And 
(unfortunately) many shops just run their people until they burn out, then 
hire freash ones, who has been thaught of the new stuff on university. 
:-(  Not mine, thanks God.

>  I don't remember the exact problem I was trying to solve (it was a
>  while back), but I do recall that the manual loop wasn't all that
>  complicated.

Perhaps it is the same old story.  The "self supporting two reasons".  1.) 
The hand-written loop is not complex, because you (and me) had worked for 
decades with them.  2.) Our way of thinking *leads* to hand-written loops, 
and preparing a problem for a generic-algorithm-based solution needs a 
*different* way of *thinking*, a different approach.

Another issue is, why do generic algorithms?  Why would we choose to create 
our own?  Because we need a *generic* algorithm.  Which can work on *any* 
iterator-range, which fits the concept it belongs to.  Now does this really 
happen often?  Do we really often need to craft algorithms for many types? 
I don't think so.

Of course, you have been talking about reusing the existing algorithms.  It 
is (as mostly always) a question of tradeoffs.  If there is no standard 
algorithm (or combinations of), which does the job for me (in a reasonably 
simple way) do I really want to go into the trouble of tweaking them?  Do I 
fall back to the problem of the previous paragraph, and create one when I 
only have *one* *very* *specific* need?

The above tradeoff we make (the design decision) is, unfortunately, not a 
fair one.  10+ years of loop-writing experience against virtually zero 
pratical experience with non-trivial algorithms.  So it is (unfortunately) 
true, that we are *inclined* to give up.  The only important thing is to get 
our resoning right.  We might have given up, because we just don't have the 
time to learn.  We might have given up, because we (ehem, I) are a bit lazy. 
Or we do not like to feel insecure.  Or because the genric algorithms of the 
STL (IMHO!!!) lack certain fundamental features and support, which would 
make them fun and easy to use *and* to understand.  Let me elaborate on that 
a bit.

The canonical "algorithm" in CA-Clipper looks like this:

(This code works on a database table, using the currently active order.)

     COUNT TO <idVar>
        [<scope>] [WHILE <lCondition>] [FOR <lCondition>]

There are *two* important details above, which are *not* provided for us, 
when using STL algorithms, but it *is* when using manual loops.  Loop 
control:

WHILE corresponds to break; in an if

FOR corresponts to continue; in an if

<scope> is our begin and end iterators.

So FOR is *filtering* elements, which we may embed into the functor, but it 
does *not* belong there.  We may also *combine* functors with some sort of 
binding, but that looks unreadable for most eyes.  But I guess lambdas (once 
supported properly by the core language to be useful without a Ph.d. degree 
in the standard) can help simplify those.

WHILE is a way to stop our looping "prematurely", without doing a find 
before the real loop (so that we know what end to tell).  This *could* be 
solved by stateful functors, but we already know that those are pain in the 
achilles region.

>  I also recall being frustrated that I spent so much time
>  on it and *still* was unable to make an algorithm do the job.

Tell me about it. :-)

>  The
>  feeling I came away with from this was that, sure, using algorithms
>  *sounds* like the right thing to do, but in practice I can write in 30
>  minutes with a manual loop something I couldn't figure out in *days*
>  using an algoritnm.  Function Objects seemed to overcomplicate the
>  code, for one thing.  For another thing, I just couldn't get the
>  algorithm to do the job I wanted done.

I believe that we need much better lambda support in the language (which in 
turn needs certain core languager support) until we can honestly say that 
non-trivial tasks can be done using a combination of generic algorithms and 
functors.  BTW the proposed extension to the langauge, which would make 
function-local classes to have linkage (and therefore be used as functors) 
would make it even nicer.

>  Herb Sutter recommends using lambda expressions from the Boost library
>  to automatically generate function objects and make algorithm calls
>  cleaner. I have looked a little at that (and will look in more detail
>  later), and it shows promise.

It does, it does show great promise, but without some sort of typeof 
(auto/decltype) support in the language it can (sometimes) be very 
cumbersome to use.  One needs to disambiguate etc.  Now being there and 
listening (and sometimes even understanding!) what is going on in 
standardization about auto/decltype I have to say, that it is a non-trivial 
task to define them well.  Minds far greater than mine have spent literally 
years on designing them, and there is still work to do.  They have to design 
featurs to be used by a programming stile (in the next decade) which does 
not exist yet!  And they must take into consideration the experienes we do 
not have yet!

>  However, I am a little skeptical because of my past experience.

Aren't we all?  But we still manage to solves those issues.  We have to 
remember, that changes that involve people take time.  A lot longer time 
than other things.  So we musn't forget that while we look at the field 
today.  As revolutionary STL was at its birth, as surely came the evolution. 
We have started to use it, and have found shortcomings.  Even in simple uses 
(recall the reference to a reference problem).  Many of those problems have 
got *nearly* perfectly fixed by the great work the Boost community has done 
over the years.  But hey could go just so far, without being able to extend 
the core language.  So there is work left, two very distinct tasks:

1.) We, the programmers, has to find the time to learn and master these new 
techniques.  We need to do that, because those who do will get the 
commercial advantage.  And we do not want to miss that train. :-)

2.) The people designing the new C++ should take all of our combined 
experiences about the weaknesses and extend the language just as much, so 
that those could be overcome.

None of the above is easy.  None of the above is painless.  And none of the 
above can be avoided.

>  I'm no genius, but I consider myself to be a
>  reasonably above average programmer.  I have a Bachelor's in CS from 10
>  years ago, have been working in the industry ever since.  I'm no
>  newbie.

That can, you know, just as well be a disadvantage. :-)  Our minds are wired 
to think in loops.  We need a paradigm shift, which a newbie does not. :-) 
So let's go out, and erase our minds with a properly large amount of 
alcoholic beverages. ;-)

>  I have a good grasp on OOP, but am also aware that it is not a
>  panacea.

While OOP might give some good ways of thinking (like encapsulation for 
exmaple) it certainly is not a whole lot of advantage in this case, as 
generic programming is a whole lot different ballgame.

>  I have written a fair bit at a low level in assembly and C,
>  but have also written a lot of Perl, Java, and of course C++.

That should already give you a lot of different viewpoints to the same 
issue.  I find my diverse language-use experiences quite rewarding in that 
manner.

>  I've
>  done a fair bit of studying on software design (including Design
>  Patterns, etc).  One of the things I *like* about C++ is that it is a
>  multi-paradigm language -- since I'm a pretty pragmatic person, and I
>  believe in knowing many things well, and using the right tool/method
>  for the right job.  Thus you could understand why I want to master the
>  whole generic programming thing.

A very commendable intention.  I say that, because I am a selfish b..rd and 
I do have the same intentions. :-)

>  Can you point me to some practical software that uses generic
>  programming in C++ extensively (perhaps a good open-source project)?

Oppps.  Not really.  But now you have just put another item to my endless 
to-do list to find one. :-)  A good one if exists.

>  Even better would be a source for a ton of examples of the nature:
>  "Here is how a programmer would probably write this using a loop.  And
>  here is the same solution using an algorithm."  I realize that you have
>  a number of similar examples in your Effective STL book.  I guess I'm
>  looking for something more extensive.

I would say that you just have successfully identified the subject of the 
next best-selling C++ books.  Any volunteers to write it? :-)

>  I obviously seem to be missing
>  something, since so many of the "top dogs" are leaning this direction.

I think it is the paradigm shift, the change in thinking.  I do believe that 
a system should be designed differently, if one wants to employ generic 
algorithms.  I also believe, that it must be designed more strictly, so that 
each concept is represented by its own distinct (user defied) type...  But 
this is rather a feeling than a proved concept.

>  But for some reason I'm not getting it.

We lack real-life examples. :-(

>  Whatever the source, it would
>  be *really* nice to find an entire resource devoted to "you *used* to
>  do it this way, using algorithms you do it *this* way".  So far I have
>  found no such resource.  Item 43 of "Effective STL" (as well as some of
>  the other items) were helpful, but I guess I just need something more.
>  Do you have any suggestions?

Unfortunately not.

>> From my perspective, there are two questions here.  One is, as I said,
> whether it is practical to replace many manual loops with algorithm
> calls.

Nope.  Not replace.  I would not start to refactor existing systems.  But to 
start to *consider* them for new code, yes.  And if they cannot be used for 
some reason, let's post it here!  Let's start to exchange our thoughts about 
the matter, until that panacea of "before using generic algorithms for two 
weeks, just 3 minutes a day" and "after" code examples come.

In this way we can also support the work of Boost and the Committee a lot: 
as they can see if we have really identified a hole on the carpet.

> The other is where people can find open-source examples of real
> systems that actually do it.

Maybe.  Not tonight though.  BTW I would start searching for such code 
around Boost, and Boost considered libraries.  Most of the large open source 
projects are either C, or are stuck (for some real and for some wrong 
reasons) in the "namespaces, templates and the STL is evil" age. :-((((

> The first question, especially, should really be answered only by people
> who spend a lot of time working on real C++ systems.

Well, I have charged a lot of hours for that.  Does that count? ;-)

> I welcome comments on both questions, as I hope to be able
> to enlighten not only my reader, but also myself.

Well, I hope that some of what I wrote is actually undertsandable, and it 
even helps a bit.

-- 
WW aka Attila
:::
Reality continues to ruin my life. - Calvin and Hobbes 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Wu Yongwei" <wuyongwei@gmail.com> writes:


[...]

| > So, this is to say that I'm highly interested in your frequent uses of
| > std::list both for my education and curiosity -- the key part i
| > "simple to explain".
| 
| I must say that when the I use the std::list container, often I am
| *not* using the list data structure.

Aha?!?

| > | Very often it is just a
| > | container of pointers to maintain some objects' lifetime.  Other
| > | containers can do too, but I find list the easiest and the fastest.
| >
| > Indeed, I'm wondering why std::vector would not have been a better
| > representation in terms of simplicity and efficiency.
| 
| Inserting at the beginning of a vector is O(n), and at the end
| amortized O(1).  For lists, they are simply O(1).

I know what the theory says.  I'm talking about the actual practice
and measurements.

-- 
                                                       Gabriel Dos Reis 
                                           gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Wu Yongwei" <wuyongwei@gmail.com> writes:

| Francis Glassborow wrote:
| > In article <m364utrimr.fsf@uniton.integrable-solutions.net>, Gabriel Dos
| > Reis <gdr@integrable-solutions.net> writes
| > >Indeed, I'm wondering why std::vector would not have been a better
| > >representation in terms of simplicity and efficiency.
| >
| > One of the interesting observations mad several years ago by Andy Koenig
| > was that whilst std::list seemed to have the edge over std::vector
| > because of the 'ease' with which items could be inserted and deleted
| > that may be based on theory rather than practice.
| >
| > He then went on to point out that with modern multi-level cache systems,
| > the contiguous nature of the vector's data storage can result in a major
| > performance gain, even an insert may be faster for small to medium size
| > data structures.
| 
| Only true if insertions (and erasures) occur at the end of the
| container.

Is it based on actual measurements?

-- 
                                                       Gabriel Dos Reis 
                                           gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Wu Yongwei wrote:

>As another poster says, you can always use `while' instead of `for'.
>You lose the elegance and compactness of expressions.

>Using STL is really an easier way for beginners to express abstract
>data structures than handmade structures.  Compare:

Yes, STL is good for education.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

James Kanze wrote:
>> I have no opinion how one should or should not like/use STL.
>> But I am also curious what's the lot part of programming one
>> can miss if not using STL? That is interesting.

>The STL entails a lot.  std::vector is the standard array type
>in C++; I would consider that someone who didn't use it didn't
>know modern C++.

Ok, let's say I don't know modern C++ in your consideration.
But what's the lot part of programming I missed by not using STL?
Tell me something different or that can be proved.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> I would say approximately half an hour, if I do it very slowly.  Compared to 
> the several days (or perhaphs weeks) it may take to design and write a 
> proper hash function.

Not in my experience. As long as you have hashing function available for 
elements, it is a job of library class to combine them.

It might not give you ideal hashing function, but it will certainly be 
good enought to outperform map in most cases.

BTW, this hash-phobia is rather funny issue. Just because STL does not 
have hash_map (yet), all STL advocates are constantly whining about how 
hard is to use hashing.

Meanwhile, other languages do use hashing as default, some even generate 
hashing function for your data automatically. Who is wrong then? All 
language and library designer other than those doing STL or those STL 
advocates whining about hashing difficulities?

Also, if you are going to use class with 101 members as key, you might 
not be as good programmer as you think you are. (And yes, it is not hard 
to create that predicate and operator== is BS, my mistake, I has never 
created ordering predicate based on operators as I think that it is 
completely stupid to perform all comparisons twice. You are right, I am 
unwilling to use wrong patterns).

> Relative to a BMW perhaps.

It is cheaper to fill random access hash map and sort it than to fill map.

>>(By stable order I mean that elements can be iterated in
>>order they entered associative container).
> 
> 
> Excuse me???  Why would I want to do that, when I want to create an 
> *ordered* container???  Map is for *ordering* *differently* than the order 
> you have entered your elements.  That is the whole point of it.

Well, it is perhaps totaly useless to explain it to you, but consider 
this example:

You have to build translator between words written in three languages 
(please, for the moment consider that this is just example, I know there 
are synonyms etc. let us consider that each word has exactly one 
counterpart in other languages).

Now if input ordering is stable, you can solve this problem this way:

struct Lang3 {
   Index<String> lang1, lang2, lang3;

   void Add(const String& l1, const String& l2, const String& l3) {
      lang1.Add(l1); lang.Add(l2); lang3.Add(l3);
   }

   String FindLang1EquivalentOfLang2(const String& l2) {
      int q = lang2.Find(l2);
      return lang1[q];
   }
};

Once again this is artifical example, but there is a lot of real-world 
situations that benefit from this feature. Meanwhile, the number of 
real-world situations that benefit from predicate ordering of map is 
rather low.

And once again, filling and sorting hash map (esp. if it is random 
access container) is cheaper than filling map in any real-world situation.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler <cxl@volny.cz> wrote:
> [...]
> > I think you have to know what operation(s) will be
> > peformed in your loops, if they will be mutating or not.
>
> Sure, if you are ideal being, you write just correct software which
> out-of box does what customer demands.

  Mhmm. For me it's rather simple: If it
  compiles with a 'const_iterator', it won't
  modify the container. If it doesn't, then
  it might.
  Where's the problem? Am I missing something?

> [...]
> Mirek

  Schobi

-- 
SpamTrap@gmx.de is never read
I'm Schobi at suespammers dot org

"Coming back to where you started is not the same as never leaving"
 Terry Pratchett



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> Well, my mistake, I should have invented something real to illustrate
> the problem. But I have to agree that this issues is much less important.

However, both loops gave a really good example for the topic of this
thread: you should not use manual loops because even simple manual
loops are buggy. Use algorithms verified to the right thing instead!
-- 
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> Real world example: Simple text editor, lines stored as std::vector<
> std::string >.

This is a brilliant example because you just put *me* into control of
the class you want to move using 'memcopy()': I have a fairly complete
own implementation of the C++ standard library. Thank you!

You cannot portably 'memcopy()' 'std::string'! A real world example is
the small string optimization which put a character buffer into the
string object itself. To avoid conditional access to the buffer in the
typical use of a 'std::string' (namely reading its value somehow), it
is natural to always have a pointer to the start of the character which
happens to be a pointer to the string object itself for small strings.
Now, if you 'memcopy()' this string somewhere you are accessing the
buffer of some other string - something which may or may not blow up
your program anytime soon. It may just settle to produce incorrect
results.

> Now to insert lines, you have to perform insert. Using my violation,
> this will be 15 times faster.

It is a recognized problem that the current STL specification has no
concept of moving objects. This is something addressed by Howard's
proposal and certain implementation techniques. Of course, since both
'std::vector' and 'std::string' are under my control, I can do as a
standard library implementer whatever I find reasonable to optimize
this. I would still not fall back to non-portable approaches for
something like this: it is bad enough to have non-portable stuff in
the implementation which cannot be avoided (e.g. the initialization of
the standard stream objects) because it is a source of contiguous
maintenance needs.

> Means that my text editor will be able to 
> reasonable work with files 15 times longer than yours (or you will be
> forced to find more complicated solution).

When reading files, it is likely that the structure containing the
the lines is extended at the end, rather than the middle. I doubt that
you measure a factor of "15" for arbitrary files sizes. Also, a different
structure than a linear container which also allows efficient insert
while allowing efficient location of areas for redraw may even be more
appropriate, e.g. a 'std::map'.

>> What is a "well-though and safe standard violation"? Something which
>> the standard states it not a violation and thus safe?
> 
> Something that is nearly impossible to break and when it breaks, it is
> nearly impossible not to notice.

I disagree. See above.

> I am not THAT good in logic, but I believe that current standard in fact
> requires some behaviour that is in accordance with my violation. In
> fact, all that I require that compiler does not change layout of object
> when constructor/non-virtual destructor is added.

False. You also require that the objects don't have internal pointers to
themselves, don't register themselves somewhere with their address, etc.

> Well, pehaps some sinister compiler writer
> could imagine a way how to spoil this effect, but I guess it is unlikely
> and maybe even impossible

You mean things like the internal pointer from the derived object to the
base object which was fashionable for the implementation of virtual base
classes? For those ABIs I'm aware of this is currently indeed not done
but it was definitely done in the past. Also, remember that the pointer
value can change when you convert it between derived and base objects.

> And above is all that I require.

Since the above is not at all everything you require, I'm even more
suspicious of your approach since it even missed a bunch of pretty obvious
restrictions.

>> because something failed miserably and caused some damage due to a
>> deliberate violation of the rules.
> 
> There is milion of things that can fail, and 999 999 is more likely to
> fail than this one.

Indeed. However, any deliberate deviation from an explicit restriction
is definitely something which you can be held liable for in a contract.
Accidental errors are normally excluded explicitly from certain kinds
of liability in contracts although even this does not always work as
intended.

> In the end, if this is about to blow, it will blow immediately.
> (Actually, you can check it easily at application startup. You do test
> your applications before shipping them, do you?)

I'm mostly writing custom application where the source code resides at
the customer or libraries distributed in source form. These are tested
at least with the system they are currently working on or all systems I
could get my hands on for testing. However, when recompiled on a
different system things may easily blow up or some subtle errors may
creep in. Any deliberate violation *will* cost me substantial amounts
if things don't work - not necessarily immediately in terms of money
but e.g. in terms of reputation or chances to get another contract.
Accidental errors also have their price but customers generally accept
that contractors are not perfect. They do not accept that contractors
are cheating for a short-term benefit.
-- 
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

White Wolf wrote:
> Like the debate in this thread talking about vector iterators being
> invalidated by a puch_back during a reallocation.  Oh my.  (Gabi's idea of
> the iterator holding a ref to the vector and an index solves this issue,
> and I guess originally it has been discarded as an idea based on the
> (proved
> evil) "efficiency" claims.)  I cannot imagine an algorithm, which would
> need
> to do that, but let's assume there is one.  So we either use deque there
> *or* any other container than the prematurely overoptimized vector and
> string.

Just a note, though: Iterators for 'std::deque<..>' are even worse than
iterators for 'std::vector<..>' because they get invalidated with every
insert (see 23.2.2.3, lib.deque.modifiers, paragraph 1). However, in
contrast to 'std::vector<..>' references and pointers to 'std::deque<..>'
elements always remain valid when inserting at the front or at the end.
The reason is that deque is similar to a list in that it allocates nodes
for its elements although each node is typically used for multiple
elements but also keeps a kind of a directory of available nodes to
facilitate random access. Iterators need to be aware of both the current
element and the directory of nodes but the latter may need increasing
when adding elements.
-- 
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> White Wolf wrote:
> > Mirek Fidler wrote:

> >>>2.) I still do not see *any* reason to change a map to a
> >>>hash, just because I am afraid of iterators.

> >>Well, let me try. What about speed?

> > What about it?  What speed?  Speed of programming?  I have a
> > class with 101 members in it (it is called
> > NineteenEightyFour), and it has a less-than operator.  I do
> > *no* lookup, I need them ordered and stay ordered.  How long
> > does it take for me to put it into a map?  5 seconds.  How
> > long does it take for me to write a proper* has function for
> > it?

> If your class has 101 members,

> a) how long it takes to you to write proper ordering predicate
> if you have operator< available for all members

Probably less time than it takes to write a proper hash function
for it.  Writing a good hashing function is non-trivial, and a
hash table with a bad hahs function is not going to be very
fast.

FWIW: I did some measurements to compare different hash
functions some time back.  I through in std::map to give me a
base line.  Some interesting data:

 -- For less than about a 100 elements, std::map was faster than
    any of the hash code implementations.

 -- Which hash code was best varied according to the data sets.

 -- One of the hash codes which did very well for most data sets
    had horrible performance for one particular data set.

As a result of my measures, for anything up to about 1000
elements, I won't even think of using anything but std::map.

--
James Kanze                                           GABI Software
Conseils en informatique orient�e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Hendrik Schober wrote:
> Mirek Fidler <cxl@volny.cz> wrote:
> 
>>[...]
>>
>>>I think you have to know what operation(s) will be
>>>peformed in your loops, if they will be mutating or not.
>>
>>Sure, if you are ideal being, you write just correct software which
>>out-of box does what customer demands.
> 
> 
>   Mhmm. For me it's rather simple: If it
>   compiles with a 'const_iterator', it won't
>   modify the container. If it doesn't, then
>   it might.
>   Where's the problem? Am I missing something?

Well, maybe. It is not about modifying elements of container, but 
container itself:

// invalid code!
for(vector<Foo>::const_iterator x = v.begin(); x != v.end(); ++x)
     if(const_condition(x))
         v.push_back(some_value);

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> > I would say approximately half an hour, if I do it very
> > slowly. Compared to the several days (or perhaphs weeks) it
> > may take to design and write a proper hash function.

> Not in my experience. As long as you have hashing function
> available for elements, it is a job of library class to
> combine them.

Do you know of any library classes that do this?  Correctly?
It's far from trivial.

> It might not give you ideal hashing function, but it will
> certainly be good enought to outperform map in most cases.

Could you please post the measurements?  I've done fairly
extensive studies of hash table performance, using various
hashing algorithms.  For tables with less that a hundred or so
elements, std::map beat all hashing algorithms I could find.  I
don't know what your statistics for "most cases" are, but in my
code, most maps contain less than a hundred elements.

> BTW, this hash-phobia is rather funny issue.

It's not phobia.  It's actual measurements.

> Just because STL does not have hash_map (yet), all STL
> advocates are constantly whining about how hard is to use
> hashing.

Defining a good hash function is not trivial.  And for smaller
tables, which are the majority in my work, std::map beats a hash
table for even the best hash function.

> Meanwhile, other languages do use hashing as default, some
> even generate hashing function for your data automatically.
> Who is wrong then?  All language and library designer other
> than those doing STL or those STL advocates whining about
> hashing difficulities?

Who knows?  I do know about the measurements I made.  In my
pre-STL library, I had a hash table, and not an ordered map.

What you can say about std::map is: it is very fast for small to
medium sized tables, and it can give absolute upper bounds given
a function which is either correct or not.  A hash table can be
significantly faster for large tables, but there is no good,
simple way of determining whether a hash function is "good" or
not.  I use std::map when it is fast enough (which is most of
the time), and only worry about having to find a good hash
function for my particular data set when it is not.

My experience with hash tables in Java is that a lot of user
defined classes forgets them; it is fairly frequent to find
classes overriding equals, but not hashCode.  While I've also
encountered classes which define < in a way not compatible with
std::map (including some of my own, for historical reasons),
globally, I've found this to be less of a problem.

> Also, if you are going to use class with 101 members as key,
> you might not be as good programmer as you think you are. (And
> yes, it is not hard to create that predicate and operator== is
> BS, my mistake, I has never created ordering predicate based
> on operators as I think that it is completely stupid to
> perform all comparisons twice. You are right, I am unwilling
> to use wrong patterns).

> > Relative to a BMW perhaps.

> It is cheaper to fill random access hash map and sort it than
> to fill map.

> And once again, filling and sorting hash map (esp. if it is
> random access container) is cheaper than filling map in any
> real-world situation.

Measurements, please.  Or at least more precisions.  That's not
my experience in the work I've done.

--
James Kanze                                           GABI Software
Conseils en informatique orient�e objet/
                    Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> > Are you saying you have written a text editor this way? Because this
> > sounds like a pretty poor approach to me, and not for the reasons you
> > state. And yes, std::list<std::string> would be a much better approach
> > (you need efficient insertion far more than you need random access
> > iterators; for display purposes you just need an iterator to the first
> > displayed line, whether it's random access or not is irrelevant),
>
> Have you ever implemented text editor in X11 or Win32? Or anything else?
> Be honest.

Condescension will not help your argument.

Yes I have, I worked on a full-featured word processor.

> I guess your are just making bold statements.

Guess again.

> list implementation is
> much more complex and certainly much slower.

You've timed it in this context?

> In fact, GUI development is always based on indicies.

Mostly because GUI APIs are typically language neutral, or at best
based on C; indices are about all you can rely on to be portable across
languages.

> You convert screen
> coordinates to lines. You get scrollbar positions as numbers. You send
> numbers as position... Only thing you could actually gain by iterators
> and list here is that you will be constantly iterating through list and
> counting screen positions. Very stupid idea.

Until someone presses return and inserts a new line, or selects a line
and deletes it.

You might want to try out "very stupid ideas" before you dismiss them
out of hand; you could be surprised.

> > but
> > still a pretty bad idea. std::deque<std::string> would probably be
>
> OMG, why? deque is totaly wrong type of container here. What you plan to
> gain using deque? Fast insertion of first line?!

Fast insertion anywhere plus those random access iterators you want.

> >>There is milion of things that can fail, and 999 999 is more likely to
> >>fail than this one.
> >>
> >>In the end, if this is about to blow, it will blow immediately.
> >
> > How do you know that? And even if it does, what will you do about it?
>
> Well, maybe I know more than you?

Maybe you do; I suppose that's possible.

> > bitten several times in the past when trying to get away with "one
> > little violation, for which I have really good reasons." What always
> > happened was that eventually the violation blew up when the system was
> > scaled up. Sometimes the scaling was due to uses I didn't anticipate
> > (like designing a system with single iheritance in mind, then using
> > multiple inheritance).
>
> Done.

Done what? You've anticipated every possible use your system could be
put to?

It's interesting that you snipped the part where I explained how
multiple inheritance broke your logic.

>   Sometimes it was due to porting it to a new
> > OS/compiler combination (sometimes even just a new version of my
> > current compiler was enough).
>
> Done.

Done what? Ported your code to every OS/compiler combination on the
planet? Including ones that haven't been created yet?

> > In every case, the standard violating
> > code/design had to be backed out and redone in a standard-conforming
> > way.
>
> Well, if some compiler writer will gone crazy, I will simply switch off
> this optimization. That is 20 minutes work.

Following the standard is going crazy?

> But I know that this simply will not happen. There is no reason. No
> possible gain.

It sounds like you do know more than me after all.

> > it won't happen to you. Maybe you won't have to grow your system in a
> > way that makes the problem intractable. Maybe you're just smarter than
> > me.
>
> Perhaps just more experienced.

Yes, it sounds like it.

> > But I find it simpler to just stay within the boundaries of
> > standard C++. It gives me one less thing to worry about.
>
> I do too. With this one single exception. I do not like to break rules,
> absolutely not. But gains are way too high to ignore here.

If they're that important to you, go for it.

Bob


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Gabriel Dos Reis wrote:
> James Kanze <kanze@none.news.free.fr> writes:

> | > That leaves me wondering whether decisions to use
> | > std::list are not often cases of premature optimisation.

> | Generally, the choice of std::list is based on the fact that
> | your code has iterators into the middle somewhere, and wants
> | to insert or delete there, without invalidating other
> | iterators.  If your algorithm is keeping iterators for
> | values, list has a decided advantage over vector or deque.

> I beg to differ.  It is not obvious that allocating the data
> somewhere else and storing their addresses in a vector is not
> a better alternative.

You mean to say that maintaining two separate data structures
might be simpler and easier than just maintaining one?

And of course, this still doesn't address the basic problem: if
I'm maintaining iterators into the list, I'm maintaining
iterators into the list.

> As I said, we spent a couple of days examining in details
> examples based on the arguments you just developed (and
> variants) and came to the conclusion that in the majority of
> the cases, std::list did not seems really appropriate,
> optimization notwithstanding.

I don't really know about the majority of the cases.  I know
that I almost never used std::list, because I almost never have
a need for a list structure in general, much less one of which I
keep iterators floating around.

I was reacting to the statement "decisions to use std::list are
not often cases of premature optimization".  I have a fairly
high opinion of the average programmer, and I presume that most
decisions to use std::list are based on its particular
characteristics.  Its most significant characters, IMHO, is the
fact that it doesn't invalidate iterators.  It's not a
characteristic that I need very often (so I don't use std::list
very often), but if I did need it, std::list seems to be a good
choice.  And of course, that decision isn't premature
optimization.

> The cases where linked list seemed to be appropriate were
> mostly complex data structures, but then the items seemed
> better organized with instrusive lists.

I tend to use intrusive lists too.  But that doesn't change the
issue: any motivation to use std::list is based on its
particular characteristics, not on some "premature
optimization".  If I can use std::list rather than an intrusive
list, I will, because it is standard.  But since the motivations
for one or the other are different (I use intrusive lists in
order to be able to remove an object from whatever list it is
in, without knowing that list), the choice is rarely present.

> I do know for sure we have such codes running here and
> elsewhere.  Another example we examined was the "pool
> allocator" idiom, as described in TC++PL3 for example.

I'm not saying that std::list should be omni-present.  Only that
when it is present, the motivations aren't necessarily premature
optimization.  Like the other containers, it has both its
advantages and its disadvantages.

--
James Kanze                                           GABI Software
Conseils en informatique orient�e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <3l67glF10vlh2U1@individual.net>, Mirek Fidler <cxl@volny.cz> 
writes
>BTW, this hash-phobia is rather funny issue. Just because STL does not
>have hash_map (yet), all STL advocates are constantly whining about how
>hard is to use hashing.

I think you mean the C++ Standard Library rather than the STL. IMO the 
STL is a set of principles for creating containers decoupled from 
'algorithms' with iterators as the interface between them. The C++ 
Standard Library provides a number of common containers and 
'algorithms'.

-- 
Francis Glassborow      ACCU
Author of 'You Can Do It!' see http://www.spellen.org/youcandoit
For project ideas and contributions: http://www.spellen.org/youcandoit/projects


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

wij@seed.net.tw wrote:
> James Kanze wrote:
> >> I have no opinion how one should or should not like/use
> >> STL.  But I am also curious what's the lot part of
> >> programming one can miss if not using STL? That is
> >> interesting.

> >The STL entails a lot.  std::vector is the standard array
> >type in C++; I would consider that someone who didn't use it
> >didn't know modern C++.

> Ok, let's say I don't know modern C++ in your consideration.
> But what's the lot part of programming I missed by not using
> STL?

Something called software engineering.  Writing code that can
easily be understood and maintained by others.  On one hand, it
means avoiding exoteria -- and that includes many of the more
exotic uses of the STL.  On the other, however, it means that
when there is a standard tool available to do the job, you use
it, and not something of your own invention.

In C++, there is a standard array type, called std::vector, and
a standard string type, called std::string.  Now, there are a
number of points in their design (particularly that of
std::string) which I don't agree with.  But they are the
standard tool; any C++ programmer reading my code knows what an
std::vector or an std::string is (or he isn't a C++ programmer).
My own String and ArrayOf are undoubtably superior (:-)), but
any C++ programmer encountering them in my code will start by
asking the simple question: why am I using them, instead of the
standard?  How are they different, so that their use is
justified, rather than just using the already present standard
component?

If you don't feel at home with some of the more exotic features
of the STL, and prefer not to use them, that's fine.  There are
parts of if I don't use either.  But if you don't use
std::string and std::vector, std::map when you need a map, along
with the everyday algorithms like copy or sort, then don't ask
me to maintain your code.

--
James Kanze                                           GABI Software
Conseils en informatique orient�e objet/
                   Beratung in objektorientierter Datenverarbeitung
9 place S�mard, 78210 St.-Cyr-l'�cole, France, +33 (0)1 30 23 00 34


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

>>OMG, why? deque is totaly wrong type of container here. What you plan to
>>gain using deque? Fast insertion of first line?!
> 
> 
> Fast insertion anywhere plus those random access iterators you want.

Since when? Insertion into arbitrary position is slower than the same 
operation with vector.

Gosh, I am not a STL user, but sometimes I feel like I know more about 
it than STL advocates....

FYI, implementing random access container with O(1) insertion is impossible.

>>Done.
> 
> 
> Done what? You've anticipated every possible use your system could be
> put to?
> 
> It's interesting that you snipped the part where I explained how
> multiple inheritance broke your logic.

No. I snipped part that I felt was dealing with library reusing and code 
refactoring.

>>>OS/compiler combination (sometimes even just a new version of my
>>>current compiler was enough).
>>
>>Done.
> 
> 
> Done what? Ported your code to every OS/compiler combination on the
> planet? Including ones that haven't been created yet?

Not every. But gone through Win32 -> Linux transition. Means in fact -> 
Posix. That rules out most of remaining possibilities.

>>Well, if some compiler writer will gone crazy, I will simply switch off
>>this optimization. That is 20 minutes work.
> 
> 
> Following the standard is going crazy?

Choosing inferior compiler implementation is crazy.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Dietmar Kuehl wrote:
> White Wolf wrote:
> 
>>Like the debate in this thread talking about vector iterators being
>>invalidated by a puch_back during a reallocation.  Oh my.  (Gabi's idea of
>>the iterator holding a ref to the vector and an index solves this issue,
>>and I guess originally it has been discarded as an idea based on the
>>(proved
>>evil) "efficiency" claims.)  I cannot imagine an algorithm, which would
>>need
>>to do that, but let's assume there is one.  So we either use deque there
>>*or* any other container than the prematurely overoptimized vector and
>>string.
> 
> 
> Just a note, though: Iterators for 'std::deque<..>' are even worse than
> iterators for 'std::vector<..>' because they get invalidated with every
> insert (see 23.2.2.3, lib.deque.modifiers, paragraph 1). However, in
> contrast to 'std::vector<..>' references and pointers to 'std::deque<..>'
> elements always remain valid when inserting at the front or at the end.
> The reason is that deque is similar to a list in that it allocates nodes
> for its elements although each node is typically used for multiple
> elements but also keeps a kind of a directory of available nodes to
> facilitate random access. Iterators need to be aware of both the current
> element and the directory of nodes but the latter may need increasing
> when adding elements.

So far for knowing what you are doing, White Wolf...

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> You cannot portably 'memcopy()' 'std::string'! A real world example is

You cannot put std::string into NTL Vector. Compile time error.

> the small string optimization which put a character buffer into the
> string object itself. To avoid conditional access to the buffer in the
> typical use of a 'std::string' (namely reading its value somehow), it
> is natural to always have a pointer to the start of the character which

Yes, I know. I am not that stupid as I might look.

>>Means that my text editor will be able to 
>>reasonable work with files 15 times longer than yours (or you will be
>>forced to find more complicated solution).
> 
> 
> When reading files, it is likely that the structure containing the
> the lines is extended at the end, rather than the middle.

?? When working with source files, it is as likely to insert lines at 
beginning, at end or in the middle.

> you measure a factor of "15" for arbitrary files sizes. Also, a different
> structure than a linear container which also allows efficient insert
> while allowing efficient location of areas for redraw may even be more
> appropriate, e.g. a 'std::map'.

Have you tried? What will be your key?

>>I am not THAT good in logic, but I believe that current standard in fact
>>requires some behaviour that is in accordance with my violation. In
>>fact, all that I require that compiler does not change layout of object
>>when constructor/non-virtual destructor is added.
> 
> 
> False. You also require that the objects don't have internal pointers to
> themselves, don't register themselves somewhere with their address, etc.

Yes. It is library-programmer contract.

> You mean things like the internal pointer from the derived object to the
> base object which was fashionable for the implementation of virtual base
> classes?

Well, to make things straight, those are my "moveable" requiremets:

- If a type is fundamental, it is moveable.

- Moveable types must not have any virtual method nor virtual bases.
Base classes (if any) and instance variables (if any) must be moveable.

- No method of moveable object can store any references to itself 
(this), its base subobjects or its instance variables to any non-local 
or static-local variable or use it in function or method call that does so.

- Moveable type has to have either pick constructor or regular copy 
constructor. This is a fallback requirement - if some sinister compiler 
writer would decide to make optimization described here impossible, 
templates requiring moveable will still be implementable.

(pick is an implicit variant of move constructor proposed by Howard)

As you see, virtual bases are explicitly disallowed.

If type satisfies this contract, programmer is allowed to mark it as 
Moveable (using struct Foo : Moveable<Foo> syntax sugar) and *only* 
*then* it can be stored in NTL Vector.

Example:

struct Foo;

Foo *global_foo;

struct Foo {
	int a;
	Foo *foo;
	int *ptr;
public:
	void Set(Foo * f) {
		foo = f;
	}
	void Ok1() {
		Foo *x = this;
	// assigning to local variable
	// -> does not prevent Foo from being moveable
	}
	void Ok2() {
		memset(&a, 0, sizeof(int));
	// memset does not store pointer
	// -> does not prevent Foo from being moveable
	}
	void Bad1() {
		foo = this;
	// assigning to non-local variable
	// -> makes Foo non-moveable
	}
	void Bad2() {
		ptr = &a;
	// assigning to non-local variable
	// -> makes Foo non-moveable
	}
	void Bad3() {
		global_foo = this;
	// assigning to non-local variable
	// -> makes Foo non-moveable
	}
	void Bad4(Foo& another) {
		another.Set(this);
	// Set stores reference
	// -> makes Foo non-moveable
	}
	void Bad5() {
		static int *q = &a;
	// assigning to static local variable
	// -> makes Foo non-moveable
	}
};

>>And above is all that I require.
> 
> 
> Since the above is not at all everything you require, I'm even more
> suspicious of your approach since it even missed a bunch of pretty obvious
> restrictions.

I mean that it is all that I require from compiler/ABI. Otherwise I am 
far more restrictive as seen above, OTOH those requirements are easily 
satisfied for most types used in compositions - all U++ containers and 
String class satisfy them (as well as any concrete class of U++). Means 
also that most user defined composite types do.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler <cxl@volny.cz> wrote:
> Hendrik Schober wrote:
> [...]
> >   Mhmm. For me it's rather simple: If it
> >   compiles with a 'const_iterator', it won't
> >   modify the container. If it doesn't, then
> >   it might.
> >   Where's the problem? Am I missing something?
>
> Well, maybe. It is not about modifying elements of container, but
> container itself:
>
> // invalid code!
> for(vector<Foo>::const_iterator x = v.begin(); x != v.end(); ++x)
>      if(const_condition(x))
>          v.push_back(some_value);

  I doubt that this would be a common problem for
  me, because I rarely ever hand around containers,
  but mostly only iterators. So this is reduced to
  containers being modified directly in the loop.
  It's hard to oversee this.

> Mirek

  Schobi

-- 
SpamTrap@gmx.de is never read
I'm Schobi at suespammers dot org

"Coming back to where you started is not the same as never leaving"
 Terry Pratchett



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Scott Meyers" <Usenet@aristeia.com> wrote in message news:MPG.1d535315dd12a43e9897e1@news.hevanet.com...
[snip]
> In summary, the empirical data suggest that there is no noticable
> advantage of algorithm calls over for-loops with regards to
> comprehention speed nor with regards to recognizing the purpose
> of a stereotypical loop.
[snip]


Some results of comparative perfomance measurement: "for-loop vs. for_each":

http://alexvn.freeservers.com/s1/perfo/tests/pftests.htm  See "for-loop vs. for_each".

-- 
 Alex Vinokur
     email: alex DOT vinokur AT gmail DOT com
     http://mathforum.org/library/view/10978.html
     http://sourceforge.net/users/alexvn






      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> Could you please post the measurements?  I've done fairly
> extensive studies of hash table performance, using various
> hashing algorithms.

Well, I guess you can consider this as real world example (I hope U++ 
stuff will not be too hard to understand):

#include <Core/Core.h>

#include <stdio.h>
#include <iostream>
#include <fstream>
#include <time.h>
#include <vector>
#include <algorithm>
#include <map>
#include <deque>
#include <string>

using namespace std;

void BenchNTL(const char *file) {
	std::ifstream in(file);
	if (!in) {
		std::cout << "Cannot open input file.\n";
		return;
	}
	
	VectorMap<String, Vector<int> > map;
	int line = 1;
	
	for(;;) {
		int c = in.get();
		if(c == EOF) break;
		if(isalpha(c) || c == '_') {
			String id;
			id += c;
			c = in.get();
			while(c != EOF && (isalnum(c) || c == '_')) {
				id.Cat(c);
				c = in.get();
			}
			map.GetAdd(id).Add(line);
		}
		else
		if(isdigit(c))
			do c = in.get();
			while(c != EOF && (isalnum(c) || c == '.'));
		if(c == '\n')
			++line;
	}

	Vector<int> order = GetSortOrder(map.GetKeys());
/* For benchmark purposes, output is omitted
	for(int i = 0; i < order.GetCount(); i++) {
		std::cout << map.GetKey(order[i]) << ": ";
		const Vector<int>& l = map[order[i]];
		for(int i = 0; i < l.GetCount(); i++) {
			if(i) std::cout << ", ";
			std::cout << l[i];
		}
		std::cout << '\n';
	}
*/
}

void BenchSTL(const char *file) {
	std::ifstream in(file);
	if (!in) {
		std::cout << "Cannot open input file.\n";
		return;
	}
	
	map< string, vector<int> > imap;
	int line = 1;
	
	for(;;) {
		int c = in.get();
		if(c == EOF) break;
		if(isalpha(c) || c == '_') {
			std::string id;
			id += c;
			c = in.get();
			while(c != EOF && (isalnum(c) || c == '_')) {
				id += c;
				c = in.get();
			}
			imap[id].push_back(line);
		}
		else
		if(isdigit(c))
			do c = in.get();
			while(c != EOF && (isalnum(c) || c == '.'));
		if(c == '\n')
			++line;
	}

/* For benchmark purposes, output is omitted
	map< std::string, vector<int> >::const_iterator e = imap.end();
	for(map< std::string, vector<int> >::const_iterator i = imap.begin(); i 
!= e; i++) {
		std::cout << i->first << ": ";
		vector<int>::const_iterator e = i->second.end();
		vector<int>::const_iterator b = i->second.begin();
		for(vector<int>::const_iterator j = b; j != e; j++) {
			if(j != b) std::cout << ", ";
			std::cout << *j;
		}
		std::cout << '\n';
	}
*/
}

CONSOLE_APP_MAIN
{
	String fn;
	int argc = CommandLine().GetCount();
	const Vector<String>& argv = CommandLine();
	if(argc < 1)
		fn = GetDataFile("main.cpp");
	else
		fn = argv[0];

	int n;

	{
		BenchSTL(fn);
		TimeStop tm;
		for(n = 0; n < 10000; n++)
			BenchSTL(fn);
		cout << "STL time: " << tm.Elapsed() << " ms \n";
	}

	{
		BenchNTL(fn);
		TimeStop tm;
		for(n = 0; n < 10000; n++)
			BenchNTL(fn);
		cout << "NTL time: " << tm.Elapsed() << " ms\n";
	}
}

The purpose of the code is to print all occurences of identifiers in 
specified file, sorted using their names.

As test case, it is applied on itself (fn = GetDataFile("main.cpp")) - 
that in turn means that number of elements will not be high (which in 
fact favors map). Note also that

	Vector<int> order = GetSortOrder(map.GetKeys());

is used to sort results for VectorMap to penalize VectorMap for 
incorrect ordering for example considered (now if I would wish to print 
results in order of first appearance, 'map' penalty would be much higher).

Using Visual C++ 7.1, it gives these results:

STL time: 2953 ms
NTL time: 2454 ms

GCC 3.4 times:

STL time: 7390 ms
NTL time: 3907 ms

Also consider that these numbers include all input stream operations and 
parsing of input file.

If you have access to win32 or linux/i386 machine, you can easily try it 
yourself, just download U++ from upp.sf.net, it is one of examples 
provided with standard installation.

> For tables with less that a hundred or so
> elements, std::map beat all hashing algorithms I could find.  I
> don't know what your statistics for "most cases" are, but in my
> code, most maps contain less than a hundred elements.

I suspect that your measurements might have been somewhat skewed by poor 
Sparc's modulo operation.

> What you can say about std::map is: it is very fast for small to
> medium sized tables, and it can give absolute upper bounds given
> a function which is either correct or not.  A hash table can be
> encountered classes which define < in a way not compatible with
> std::map (including some of my own, for historical reasons),
> globally, I've found this to be less of a problem.

Well, in fact, I know how to implement similar random access tree maps, 
I just would have to find a good reason for doing so.... IME, random 
access hash-maps performs pretty predictably, regardless number of 
elements stored. That is why I favor them in any case - even if for some 
very small number of elements map would be faster, using it in 
implementation does not scale well.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
>> I would say approximately half an hour, if I do it very slowly.
>> Compared to the several days (or perhaphs weeks) it may take to design
>> and write a proper hash function.
>
> Not in my experience. As long as you have hashing function available for
> elements, it is a job of library class to combine them.

As long as... I do *not* have hashing functions available.  And you know 
why?  Because I do *not* need hashing, and I refuse to complicate my design 
for absolutely no reason.

> It might not give you ideal hashing function, but it will certainly be
> good enought to outperform map in most cases.

That is your *opinion*.  But since hashing gives me *no* ordering, I could 
not care less that it does *something* *else* than I need, faster.

> BTW, this hash-phobia is rather funny issue.

Excuse me?  We are discussing *your* iterator phobia.  I do use hashes. 
Where I *need* them.  And I am afraid I have a lot of experiences , which 
prove that I do *not* want to use them, where a do not need them.

> Just because STL does not
> have hash_map (yet), all STL advocates are constantly whining about how
> hard is to use hashing.

1.) I am no STL advocate.

2.) I am not against using hashes

I would *really* appreciate if you could stick to the topic we discuss here. 
I *hate* when someone changes the topic and tries to prove concept A by 
saying that concept B is true.  I can understand that we have found here 
your second fixa idea, but I refuse to discuss it any further, as:

1.) it is *not* the topic of this thread

2.) you are making it a question of personality flaw rather than a question 
of technical expertise

> Meanwhile, other languages do use hashing as default, some even generate
> hashing function for your data automatically. Who is wrong then? All
> language and library designer other than those doing STL or those STL
> advocates whining about hashing difficulities?

I really really ask the moderator to please *reject* all such crap in the 
future, what we see above.  Here, a person who is whining about iterators 
for a week now, who cannot draw *one* hand-written loop without at least one 
serious error in it is allowed to make personal remarks about people, people 
who actually *know* what they are doing?

I am sorry, but what *technical* statement or proof is in the above 
paragraph?  What else is there than remarks about personalities without a 
single word of proof or an attempt to give a technical reason?

> Also, if you are going to use class with 101 members as key, you might
> not be as good programmer as you think you are.

Well, or may be it is you, who has to start accepting: there is more under 
heaven and earth than what you have met alone?

> (And yes, it is not hard
> to create that predicate and operator== is BS, my mistake, I has never
> created ordering predicate based on operators as I think that it is
> completely stupid to perform all comparisons twice. You are right, I am
> unwilling to use wrong patterns).

Nope.  You are unwilling to use any pattern.  And you build a new world 
order of excuses and justifications to get away with it.

>> Relative to a BMW perhaps.
>
> It is cheaper to fill random access hash map and sort it than to fill
> map.

And perhaps filling and sorting just does not fit the bill?  Is filling and 
updating a map more expensive, than filling, sorting a hash, and then 
insert, sort, insert, sort, insert, sort, 200 times per second?  Is it? 
Ever thought about that?  Ever though about it, that there might be *other* 
requirements under the sun?

You seem to be a programmer who works in a *very* limited field, knows a 
*very* limited usage scennario, but feels no shame to try to force his 
limited opinions on every other (unknown) scenario.

>>> (By stable order I mean that elements can be iterated in
>>> order they entered associative container).
>>
>>
>> Excuse me???  Why would I want to do that, when I want to create an
>> *ordered* container???  Map is for *ordering* *differently* than the
>> order you have entered your elements.  That is the whole point of it.
>
> Well, it is perhaps totaly useless to explain it to you, but consider
> this example:
>
> You have to build translator between words written in three languages
[SNIP]

Yes, it is completely useless.  I have not need to build a translator.

-- 
WW aka Attila
:::
Stand for something, or you'll fall for anything. 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
[SNIP]
> The purpose of the code is to print all occurences of identifiers in
> specified file, sorted using their names.
[SNIP]

And as such, it has absolutely no relevance to the work I do - or IMHO most 
of the work anyone does.  It is a singled out, special case.  Let me quote 
from my old and kind lab teacher from highschool:

"One measurement is no measurement.  Two measurements are a halp measurement 
etc."

How do you expect to be taken seriously, when you are benchmarking with 
*one* case, with *very* short data which does not resemble *any* real life 
input-distribution (referring back to hash functions and also what Gabi 
wrote about them as well).

I mean I do not want to sound rude, but this starts to become ridiculous. 
Please take those AOP books and read a little.  I, who do not know much 
about hashes, already see the *huge* holes in your reasoning.  And those who 
do seem to see the same - and more.

-- 
WW aka Attila
:::
Only dead fish go with the flow. 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <3l6jbeF114h6eU1@individual.net>,
 Mirek Fidler <cxl@volny.cz> wrote:

> 			imap[id].push_back(line);

An interesting addition to NTL might be something to mimic:

    imap[std::move(id)].push_back(line);

Perhaps something along the lines of:

    map.GetAddPick(id).Add(line);

In this example, id is about to be destructed anyway, so you might as 
well pilfer resources from it if you can.

Admittedly in this example it isn't going to make a tremendous 
difference (increase in speed was barely measurable by my experiments 
here).  I attribute that to the fact that I'm using the short string 
optimization, and 77 out of 80 strings are short in my main.cpp (moving 
short strings is close to the same speed as copying them).

Fwiw, I modified BenchNTL to instead use Metrowerks::hash_map and got a 
nearly identical ratio of times that you show with Visual C++ 7.1.

-Howard

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Scott Meyers wrote:
> >From my perspective, there are two questions here.  One is, as I said,
> whether it is practical to replace many manual loops with algorithm calls.
> The other is where people can find open-source examples of real systems
> that actually do it.

I find many algorithms tremendously useful: stable_sort, equal_range,
etc....  However, I find that the cost of creating a functor far away
from the place it is actually used is almost always a detriment to
readability.  Sure, Boost has lambda operations, but they aren't
standard, and not everyone knows them.  My rules of thumb are this:

* If the algorithm involves a simple iterative loop through all
elements in a range (for_each, transform, etc...), then I roll my own
for loop.  I find that others find this much easier to read --
especially since they don't have to go looking up how a given functor
works internally.

* Otherwise, use an STL algorithm or develop my own and use it (it may
come in handy later).

* Lastly, the above "rules of thumb" shouldn't be held to too strictly
-- they are just that.  It is more important to make the code readable
and maintainable than it is to follow rules of thumb.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler <cxl@volny.cz> wrote:
> [...]
> So far for knowing what you are doing, White Wolf...
>
> Mirek

  I'd consider this a personal attack and find it
  disturbing that things like this make it through
  moderation.

  Schobi

-- 
SpamTrap@gmx.de is never read
I'm Schobi at suespammers dot org

"Coming back to where you started is not the same as never leaving"
 Terry Pratchett



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler <cxl@volny.cz> wrote:
> [...]
> > Done what? Ported your code to every OS/compiler combination on the
> > planet? Including ones that haven't been created yet?
>
> Not every. But gone through Win32 -> Linux transition. Means in fact ->
> Posix. That rules out most of remaining possibilities.

  Already herding more than just two platforms, we
  felt on the safe side when we recently ported a
  major app to yet another one. Nevertheless, we
  ran into UB that manifested itself desastrous on
  the new platform, while it behaved as expected
  by the programmer(s) on all the others.

> [...]
> Mirek

  Schobi

-- 
SpamTrap@gmx.de is never read
I'm Schobi at suespammers dot org

"Coming back to where you started is not the same as never leaving"
 Terry Pratchett



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> Hendrik Schober wrote:
> > Mirek Fidler <cxl@volny.cz> wrote:
> >
> >>[...]
> >>
> >>>I think you have to know what operation(s) will be
> >>>peformed in your loops, if they will be mutating or not.
> >>
> >>Sure, if you are ideal being, you write just correct software which
> >>out-of box does what customer demands.
> >
> >
> >   Mhmm. For me it's rather simple: If it
> >   compiles with a 'const_iterator', it won't
> >   modify the container. If it doesn't, then
> >   it might.
> >   Where's the problem? Am I missing something?
>
> Well, maybe. It is not about modifying elements of container, but
> container itself:
>
> // invalid code!
> for(vector<Foo>::const_iterator x = v.begin(); x != v.end(); ++x)
>      if(const_condition(x))
>          v.push_back(some_value);

It's hard to look at this as anything but sloppy code. I agree with
Attila and others, it's not unreasonable to expect the programmer to
know what he's doing. And in any case, debugging standard libraries
will catch this error right away.

Bob


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> >>OMG, why? deque is totaly wrong type of container here. What you plan to
> >>gain using deque? Fast insertion of first line?!
> >
> > Fast insertion anywhere plus those random access iterators you want.
>
> Since when? Insertion into arbitrary position is slower than the same
> operation with vector.
>
> Gosh, I am not a STL user, but sometimes I feel like I know more about
> it than STL advocates....
>
> FYI, implementing random access container with O(1) insertion is impossible.

D'oh! You're right; I wrote that late at night and was a little sleep
deprived.

I'll cut to the chase: here's how the word processor I mentioned
worked. The text document data structure was not a vector, deque, or
list, and std::string was nowhere in sight either. The document was an
array of pointers to character buffers. Each buffer was allowed to grow
to around 1500 characters; any buffer that grew too large would be
split into two buffers. There was no attempt to align buffer endings
with line endings; because of word wrap and the fact that the user
could change the margins, doing that was pointless. There was also an
auxilliary data structure that was essentially an array of pointers
that pointed to the beginnings of lines; this data structure would be
rebuilt whenever the user changed margins, and would be modified as the
user entered text. For display purposes, all that was necessary was the
index into the line array of the highest line that interesected the
display. Yes, this meant that scrolling involved searching back and
forth through the text, but since lines could be different heights it
was unavoidable, and in practice worked just fine, with no noticable
performance problems.

The main reason no std types were used was that this was all done in C;
if it were done in C++, I can see std::vector being used. No way would
std::string be used.

> >>Done.
> >
> > Done what? You've anticipated every possible use your system could be
> > put to?
> >
> > It's interesting that you snipped the part where I explained how
> > multiple inheritance broke your logic.
>
> No. I snipped part that I felt was dealing with library reusing and code
> refactoring.

Really? You don't remember the part where you said "a pointer to a T1
type derived from a T struct means the T1 and the T are laid out the
same way" and I said "no, here's a multiple inheritance
counterexample"?

> >>>OS/compiler combination (sometimes even just a new version of my
> >>>current compiler was enough).
> >>
> >>Done.
> >
> > Done what? Ported your code to every OS/compiler combination on the
> > planet? Including ones that haven't been created yet?
>
> Not every. But gone through Win32 -> Linux transition. Means in fact ->
> Posix. That rules out most of remaining possibilities.

What has Posix got to do with anything? We're talking about compilers,
and in any case Posix and C++ have nothing to say about each other.

> >>Well, if some compiler writer will gone crazy, I will simply switch off
> >>this optimization. That is 20 minutes work.
> >
> > Following the standard is going crazy?
>
> Choosing inferior compiler implementation is crazy.

So a compiler implementation that breaks your code is crazy?

Like I said, if the performance gains are worth it for you, go for it.
But your arguments are somewhat less than convincing to me.

Bob


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Dietmar Kuehl wrote:
> White Wolf wrote:
>> Like the debate in this thread talking about vector iterators being
>> invalidated by a puch_back during a reallocation.  Oh my.  (Gabi's idea
>> of the iterator holding a ref to the vector and an index solves this
>> issue, and I guess originally it has been discarded as an idea based on
>> the (proved
>> evil) "efficiency" claims.)  I cannot imagine an algorithm, which would
>> need
>> to do that, but let's assume there is one.  So we either use deque there
>> *or* any other container than the prematurely overoptimized vector and
>> string.
>
> Just a note, though: Iterators for 'std::deque<..>' are even worse than
> iterators for 'std::vector<..>' because they get invalidated with every
> insert (see 23.2.2.3, lib.deque.modifiers, paragraph 1).

Sure.  But we talked about push_back in this case.

> However, in
> contrast to 'std::vector<..>' references and pointers to 'std::deque<..>'
> elements always remain valid when inserting at the front or at the end.

Exactly, and push_back was the original topic.  If you insert anything into 
a vector in the middle, inside an iterating loop, your index/iterator will 
also be invalidated, if you happen to insert before or at the position where 
you are.


-- 
WW aka Attila
:::
There are no passengers on spaceship Earth - we are all the crew. 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

kanze@gabi-soft.fr writes:

| Gabriel Dos Reis wrote:
| > James Kanze <kanze@none.news.free.fr> writes:
| 
| > | > That leaves me wondering whether decisions to use
| > | > std::list are not often cases of premature optimisation.
| 
| > | Generally, the choice of std::list is based on the fact that
| > | your code has iterators into the middle somewhere, and wants
| > | to insert or delete there, without invalidating other
| > | iterators.  If your algorithm is keeping iterators for
| > | values, list has a decided advantage over vector or deque.
| 
| > I beg to differ.  It is not obvious that allocating the data
| > somewhere else and storing their addresses in a vector is not
| > a better alternative.
| 
| You mean to say that maintaining two separate data structures
| might be simpler and easier than just maintaining one?

Only if you equate "better alternative" with "easier" and
think of an allocator as maintaining a separate data structure --
which a std::list does anyway.

| And of course, this still doesn't address the basic problem: if
| I'm maintaining iterators into the list, I'm maintaining
| iterators into the list.

Woaw.

| > As I said, we spent a couple of days examining in details
| > examples based on the arguments you just developed (and
| > variants) and came to the conclusion that in the majority of
| > the cases, std::list did not seems really appropriate,
| > optimization notwithstanding.
| 
| I don't really know about the majority of the cases.  I know
| that I almost never used std::list, because I almost never have
| a need for a list structure in general, much less one of which I
| keep iterators floating around.

Ah?

| I was reacting to the statement "decisions to use std::list are
| not often cases of premature optimization".

I have not seen that statement.  Which message did I miss?

| I have a fairly
| high opinion of the average programmer, and I presume that most
| decisions to use std::list are based on its particular
| characteristics.  

Which are?  The point you seem to miss is that I'm talking of uses of
std::list where it is usually believed to be better alternative
based on a priori evaluation, not evaluation with concrete practice
and data to support the decision.  Someone in this thread already
lectures on the theoretical complexity of list over vector, while I
was talking about actual practice and evaluation in context.

-- 
                                                       Gabriel Dos Reis 
                                           gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Gabriel Dos Reis wrote:
> "White Wolf" <wolof@freemail.hu> writes:
[SNIP]
>>> That is an interesting question, but I have not seen people suggest
>>> that all iterators shall be banished.
>>
>> I did.
>
> Please do enlighten me.

http://tinyurl.com/7l4yy

aka

http://groups-beta.google.com/group/comp.lang.c++.moderated/msg/b5d81eb0789da0e7?hl=fr&


Second paragraph, if you consider the "IME, no" as the first one.

>>> At the extreme, one can consider an integer value as a canonical form
>>> of iterator -- if you think of a sequence as a mapping from |N to
>>> <value>, then the domain of the mapping is a set of iterators.  Of
>>> course, there are various irregulatiries that do not feet nicely in
>>> that picture, but most well-behaved sequences do.
>>
>> It is (IMHO) a simplification, which has no reason to exist, other than
>> old habbits.
>
> "old habbits"?  Please explain.

Yep.  Thinking in idexes.  Its like when you say: I saw Claudia Schiffer on 
the teli.  You have seen electrons hitting and illuminating a special 
surface behind thick glass.  We man are simple, and it is enough.  But if we 
start to talk about the CRT itself, we have to realize that it is not the 
picture: it just creates it.

Same with iterators and indices.  Iterators are *not* *the* order.  They 
*provide* *an* order.  But an index (if we both understand the same when we 
say this term) *is� the order.  It is an ordinal number.

Iterators are not ordinal numbers and not even ordinal (unless they are 
random access ones, which are in fact indices in disguise).  They provide an 
order.  And the "ordinal number" does not even show up in our code.  When I 
am on element 5, I am there, because I have 5 times "stepped" my iterator 5 
times.  The usual implementation of the standard map is a good example for 
that.  It does *not* have a 5th element.  It is an rb-tree.  The iterators 
render our ordered view (walking the tree by some algorithm), which results 
us to be able to use them to get that order.

> After I sent my message, I realized -- while looking for something
> else -- that the above view is already stated in Alexander Stepanov's
> works
>
>   http://www.stepanovpapers.com/
>
> Please, explain me why they are "old habbits", what the "new habbits"
> are, what are the essential differences and the advantages of the "new
> habbits" over the "old habbits".

Well, I am not a well educated person, no masters degree, my math was only 
average, so I do not think I am able to explain it any better than the 
above.  Iterators aren't ordered/ordinal.  They provide/implement an order. 
I really feel silly, but I am unable to explain it better. :-(

-- 
WW aka Attila
:::
Yoda of Borg are we: Futile is resistance. Assimilate you, we will. 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

k...@gabi-soft.fr wrote:
>Something called software engineering.  Writing code that can
>easily be understood and maintained by others.

I am focusing on the question: "But what's the lot part of
programming I missed by not using STL?"

Your software engineering issue may be a plausible candidate.
Given plain code fragmenet:

 const size_t SLEN(5);
 char str1[SLEN],str2[SLEN];

 // copy
 for(int i=0; i<SLEN; ++i) {
   str2[i]=str1[i];
 }

 // qsort
 myQSort(str1,SLEN);

Should't it be expected maintained by any programmer?
If it could cause maintainence problem because of not being
STL based, then what am I expected to say or think?
On the contrary, insisting on STL (for all programs) can
cause numerous problems (owing from its usage requirements),
not to mention the engineering issue which concerned you.
Exotic programs can be annoying, normally from bad programs,
hence is not in the consideration, and it could be using
STL, too.

BTW, STL also belongs to the set of "exotic programs",
just that specific exotics was made Standard Template Library.
Should we support? Yes. But the reality makes supporting it
expensive.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Howard Hinnant wrote:
> In article <3l6jbeF114h6eU1@individual.net>,
>  Mirek Fidler <cxl@volny.cz> wrote:
> 
> 
>>			imap[id].push_back(line);
> 
> 
> An interesting addition to NTL might be something to mimic:
> 
>     imap[std::move(id)].push_back(line);
> 
> Perhaps something along the lines of:
> 
>     map.GetAddPick(id).Add(line);

Well, you might be right. Anyway, as you know, things are a little bit 
more complicated for me - using implicit move semantics....

That said, so far we have "raw pick" types, "pick with optional deep 
copy" types, but even if we formally defined "regular copy with optional 
pick" types, we never really get to optimizing container classes for 
them. But actually, it is a good idea. Something to consider during next 
iteration.

That said, our String implementation favors single threaded code and is 
therefore implemented as COW, so such optimization will probably gain a 
little here for single-threaded.

> Fwiw, I modified BenchNTL to instead use Metrowerks::hash_map and got a 
> nearly identical ratio of times that you show with Visual C++ 7.1.

Yes, hash_map is in the end hash_map :) But yours lacks random access 
interface... :))

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
[SNIP]
> So far for knowing what you are doing, White Wolf...

Which you were unable to detect yourself (as you are unable to write a for 
loop without less than 2 bugs).  But you are quick to make your personal 
remarks.  Which kindergarden do you attend?

-- 
WW aka Attila
:::
Where the needs of the world and your talents cross, there lies your
vocation. - Aristotle 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

White Wolf wrote:
>
> One cannot learn new tricks under the pressure of commercial software
> development.  The pace is so high, that it is not possible.

One cannot keep up with pace if doesn't learn new tricks.
That's why we participate in usenet discussions and buy new books.:)

  And
> (unfortunately) many shops just run their people until they burn out, then
> hire freash ones, who has been thaught of the new stuff on university.

I guess such shops does not last for too long.
Novice programmers are not experienced, and must at least have some
period to learn something from experienced programmers.

Greetings, Bane.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> So far for knowing what you are doing, White Wolf...

Well, I have seen false claims by most people in this forum. Since I
have seen the claim that iterators stay valid in deque repeated more
than once I thought it reasonable to correct it. For example, some other
people claim that they can portably move non-PODs with memcopy() which
is not guaranteed by the standard even if there are some restrictions
imposed on the types. So, it is not that unusual that people make false
claims. ... and I'm pretty sure that deque's iterators actually stay
valid most of the time after inserting a new element. It just fails
sometimes.
-- 
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Bob Bell wrote:
> Mirek Fidler wrote:
>
>>>>OMG, why? deque is totaly wrong type of container here. What you plan to
>>>>gain using deque? Fast insertion of first line?!
>>>
>>>Fast insertion anywhere plus those random access iterators you want.
>>
>>Since when? Insertion into arbitrary position is slower than the same
>>operation with vector.
>>
>>Gosh, I am not a STL user, but sometimes I feel like I know more about
>>it than STL advocates....
>>
>>FYI, implementing random access container with O(1) insertion is impossible.
>
>
> D'oh! You're right; I wrote that late at night and was a little sleep
> deprived.
>
> I'll cut to the chase: here's how the word processor I mentioned
> worked. The text document data structure was not a vector, deque, or
> list, and std::string was nowhere in sight either. The document was an
> array of pointers to character buffers. Each buffer was allowed to grow
> to around 1500 characters; any buffer that grew too large would be
> split into two buffers. There was no attempt to align buffer endings
> with line endings; because of word wrap and the fact that the user
> could change the margins, doing that was pointless. There was also an
> auxilliary data structure that was essentially an array of pointers
> that pointed to the beginnings of lines; this data structure would be

Well, that is one of possible implementation. Anyway, the part about
"array of pointers pointing to the beginnings of lines" sounds quite
similar to "vector<string>". Not the same thing, but quite close - just
thing that you would have one buffer per one line and you are there :)

Of course, it seems like your implementation had to deal with somewhat
different requirements, essentially since you mention "word-wrap", it
was perhaps rather paragraph based rather than lines based.

Anyway, quite equivalent solution here is vector<paragraph>...

> The main reason no std types were used was that this was all done in C;
> if it were done in C++, I can see std::vector being used. No way would
> std::string be used.

It depends. If there is no furhter formating involved, string is good
both for line or paragraph version.

In my recent wordprocessor engine, capable of full formating (including
nested tables), it is Vector<Part>, and this Part is quite complicated
structure that can (if Part is paragraph) contains String which in turn
contains serialized content of paragraph. It can also contain
information about table - if that is true, that table information
contains grid of RichText cells - RichText represents text - nested
tables obviously need recursive data model.

If you are interested in overviewing this code, you can download it from
upp.sf.net (download U++, install and go for RichText package. You can
try it using "UWord" example. And, btw, at the moment it is not
optimized for extremly long texts, but that really has nothing to do
with containers used, just time constraints for development - I am not
speking about simple lines editor. But than again, you can try editor
based on Vector<String> as described above in main TheIDE window...).

>>No. I snipped part that I felt was dealing with library reusing and code
>>refactoring.
>
>
> Really? You don't remember the part where you said "a pointer to a T1
> type derived from a T struct means the T1 and the T are laid out the
> same way" and I said "no, here's a multiple inheritance
> counterexample"?

I missed it :) I will look back and respond.

> What has Posix got to do with anything? We're talking about compilers,
> and in any case Posix and C++ have nothing to say about each other.

Well, please read carefuly my  otherpost to Dietmar explaining my real
requirements and arrangement of things.

>>>>Well, if some compiler writer will gone crazy, I will simply switch off
>>>>this optimization. That is 20 minutes work.
>>>
>>>Following the standard is going crazy?
>>
>>Choosing inferior compiler implementation is crazy.
>
>
> So a compiler implementation that breaks your code is crazy?

Spoiling my optimization is impossible without compromising quality of
generated.

Only compiler that will emit data memebers to the layout of non-virtual
class just because it has constructor or destructor can blow my moveable up.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> Again, nope; what the standard says is that if you have a pointer to a
> T1 and convert it to a pointer to a T, the conversion works. It doesn't
> say anything about T1 being laid out the same way as the T. For
> example, a T1 could have data before any of the T members, which would
> be the case with:
>
> struct B {
>    int x;
> };
>
> class T1 : public B, public T {
> };

Well, but I think this is not the point.

The point is that for each class (that otherwise satisfies my
requirements) you can create POD by removing constructor/desctructor.

so things would rather be arranged this way:

class Tm : public B, public T {};

class T : Tm {
         T();
        ~T();
};

critical is difference between T and Tm layout.

Well, to calm you down, I might consider adding "no multiple
inheritance" to my requirements. Or maybe "no inheritance" will be even
better. I never used it with any derived class anyway (actually, derived
classes are not what you are usually storing into vector...).

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Scott Meyers wrote:
> I got the following from Bjorn Reese:
>
> One of the "selling-points" of algorithm calls is that each call has a
> descriptive name that indicates the purpose of the call. So, if you use
> find_if(), then others can immediately see that you are looking for the
> first element that fits a given criterion. For example:
>
>      nail = find_if(haystack.begin(), haystack.end(), isNail);
>
> Now, for this to be a real selling-point, it has to claim either that
> algorithm calls are comprehended more quickly than for-loops, or that
> programmers have more difficulty recognizing the purpose of an
> equivalent for-loop like:
>
>      for (nail = haystack.begin(); nail != haystack.end(), ++nail)
>        if (isNail(*nail))
>          break;
>
> I recently conducted a study that, amongst others, investigates this
> claim.

I think that there is a related, and probably more important, claim to
be made, which is roughly the inverse of the claim discussed here.

Programmers, especially with a couple of years of experience reading
and writing the language, are very good at recognizing stereotypical
loops -- arguably *too* good.  That is, if something looks like a
stereotypical loop, it may be recognized as such...even if it really
isn't one.  "Faux" trivial loops can arise in a number of ways,
including error (it was *meant* to be a trivial traversal...but it was
mis-coded) and through code maintenance (it *was* a trivial loop, but a
little extra functionality was added by another programmer).  In cases
like these, cognitive "chunking" can actually get in the way of
actually reading what the code really does.

Unless I go through some clever gymnastics (most of which would leave
visible traces at the call-site, even if the functor definition isn't
readily visible), a call to std::for_each *really is* a trivial
traversal.  I can't make it something else, either by mistake or
intentionally.

Any loop that can't be replaced by for_each (or find_if, or what have
you) is probably doing something tricky...and I'd better read it
closely.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Geoff Carlton wrote:

>>>count_if (names.begin(), names.end(), longer_than("Carlos"));
>>> [...]
>
> Even this simple example isn't particularly easy to understand.  Is it
> counting instances where n.length > "Carlos", or where "Carlos" >
> n.length?

Oh my!!  You don't find "longer than 'Carlos'" easy to understand??
In fact, a quick glance of the whole line reads like "count names longer
than Carlos"... In the same oprder (replacing count_if with count, and
the whole names.begin(), names.end() with "names")  If you really really
think the above is not "as obvious as it gets", then I think you are in
far more serious trouble than using or refusing STL algorithms...

Carlos
--

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Francis Glassborow wrote:
> In article <1122791044.482878.125080@o13g2000cwo.googlegroups.com>, Wu
> Yongwei <wuyongwei@gmail.com> writes
> >> He then went on to point out that with modern multi-level cache systems,
> >> the contiguous nature of the vector's data storage can result in a major
> >> performance gain, even an insert may be faster for small to medium size
> >> data structures.
> >
> >Only true if insertions (and erasures) occur at the end of the
> >container.
>
> I think you miss the point. On a system where all memory accesses take
> the same time the theoretical analysis for time to insert favours a list
> as long as you do not have to traverse the list to find the insertion
> point (which is actually quite a big if). However modern systems have at
> least three levels of storage (cache, main memory and backing store --
> used as virtual memory) and many have five or more levels (with three
> levels of cache).

No, I did not.  While your statement could be true in some cases where
the class is trivial and the vector is small, I fail to see how a
memory cache miss (discussion of virtual memory excluded) can compare
to O(n) complexity of memory copies, copy-constructions, and
destructions in the general case.

Do you call it a premature optimization if a performance-sensitive
application takes into account the algorithmic complexity?  I will not.

> If the whole of a vector fits in the first level data cache it is quite
> likely that it outperforms a list where the nodes are scattered.
>
> >
> >> That leaves me wondering whether decisions to use std::list are not
> >> often cases of premature optimisation.
> >
> >Even if insertions and erasures occur at the end of vector, std::list
> >will ensure predictability, an often-needed feature that is only
> >exhibited by languages like C and C++.
>
> Predictability? I assume you are referring to objects remaining at the
> same address. That is also true of a deque.

Of course not.  I meant predictativity in timing as in real-time
systems.

> I will repeat, choice of sequence container is often an optimisation.
> Experience teaches us that optimisation is not something that we get
> intuitively right, not least because our decisions are too often based
> on a false view of the way a system works in practice.

I will repeat, it is just ridiculous if I tell other people to ignore
algorithmic complexity.

Best regards,

Yongwei


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> > Inserting at the beginning of a vector is O(n), and at the end
> > amortized O(1).  For lists, they are simply O(1).
>
> There is deque.

Deque is amortized O(1), not O(1) (though I might have paid a fair
amount of attention to deque before).

Best regards,

Yongwei


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> > Even if insertions and erasures occur at the end of vector, std::list
> > will ensure predictability, an often-needed feature that is only
> > exhibited by languages like C and C++.
>
> Perhaps on CP/M machine. There is zillion of things that can happen and
> completely spoil predictability. Cache line misses, allocator issues,
> pipeline stalls, page swaps...

Only true on non-real-time machines.  For our quasi-real-time
applications on x86 hardware, at least we will turn off virtual memory.

> Performance today is stochastic thing. Never expect, always measure.

You cannot measure all things beforehand, and you cannot measure on all
possible platforms with all possible combinations.  Apart from that,
data structures are not necessarily easy to change when the system is
up already.

Best regards,

Yongwei


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

>>Well, maybe. It is not about modifying elements of container, but
>>container itself:
>>
>>// invalid code!
>>for(vector<Foo>::const_iterator x = v.begin(); x != v.end(); ++x)
>>     if(const_condition(x))
>>         v.push_back(some_value);
>
>
> It's hard to look at this as anything but sloppy code. I agree with

Well, the code was presented to showcase the fact that the problem has a
little to do with const or non-const iterator.

> Attila and others, it's not unreasonable to expect the programmer to
> know what he's doing.

The code written exactly as it is here is indeed rare.

The thing is that many classes that use vector in implementation have to
store 'positions' in the vector. Now thing to decide is whether those
positions are to be stored as iterators or indicies. IME it is not a
good idea to store vector iterators into non-local variables for exactly
this reason (that is the actual point). BTW, that does not mean you can
be totaly careless about positions stored as you mutate your vector. It
is just that with indicies things seem to be more straight.

E.g. our favorite example, you insert the line in the editor, you have
to check whether some stored line indicies are past the line inserted
and increment them. With iterators that would mean to obtain offset
(actually line index) before insert is performed, then insert, then
recalculate new iterators. IMHO more error-prone and certainly more work
to do.

Then, as your positions have to be stored as indicies anyway, it seems
to me annoying to always convert them to iterators and back for every
iteration over vector. As I encounter similar situations in 60% of
cases, I simply decided to implement the remaining 40% of loops the same
way.

> And in any case, debugging standard libraries
> will catch this error right away.

Actually, this is the problem. Code like this might not do anything
wrong for months (because iterators get invalidated only when vector's
internal buffer resizes). This is the worst kind of bugs.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> than once I thought it reasonable to correct it. For example, some other
> people claim that they can portably move non-PODs with memcopy() which

I hope that it will to be too rude and too off-topic to ask you for 
reading about what I really do with memcpy and non-PODs (my last post to 
you). I have spent a lot of time creating that detailed post :)

Just for the record, I do not claim that you can portably move non-PODs.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Gabriel Dos Reis wrote:
> "Wu Yongwei" writes:
>
> | Francis Glassborow wrote:
> | > In article <m364utrimr.fsf@uniton.integrable-solutions.net>, Gabriel Dos
> | > Reis <gdr@integrable-solutions.net> writes
> | > >Indeed, I'm wondering why std::vector would not have been a better
> | > >representation in terms of simplicity and efficiency.
> | >
> | > One of the interesting observations mad several years ago by Andy Koenig
> | > was that whilst std::list seemed to have the edge over std::vector
> | > because of the 'ease' with which items could be inserted and deleted
> | > that may be based on theory rather than practice.
> | >
> | > He then went on to point out that with modern multi-level cache systems,
> | > the contiguous nature of the vector's data storage can result in a major
> | > performance gain, even an insert may be faster for small to medium size
> | > data structures.
> |
> | Only true if insertions (and erasures) occur at the end of the
> | container.
>
> Is it based on actual measurements?

OK, I think some real measurements are needed, both for you and for
myself.

The test program is at:

http://wyw.dcweb.cn/test.cpp
http://wyw.dcweb.cn/rdtsc.h

(Non-x86 programmers should replace the rdtsc part for timing;
otherwise it should work on other platforms.)

The results on a Pentium 4 Linux box:

    test_deque_pod_ins_front() takes 2880 cycles
    test_deque_pod_ins_front() takes 4524 cycles
     test_list_pod_ins_front() takes 18144 cycles
     test_list_pod_ins_front() takes 12116 cycles
   test_vector_pod_ins_front() takes 107664 cycles
   test_vector_pod_ins_front() takes 52120 cycles
    test_deque_str_ins_front() takes 131804 cycles
    test_deque_str_ins_front() takes 92148 cycles
     test_list_str_ins_front() takes 101764 cycles
     test_list_str_ins_front() takes 101008 cycles
   test_vector_str_ins_front() takes 2208680 cycles
   test_vector_str_ins_front() takes 2170640 cycles
     test_deque_pod_ins_back() takes 24488 cycles
     test_deque_pod_ins_back() takes 2940 cycles
      test_list_pod_ins_back() takes 20544 cycles
      test_list_pod_ins_back() takes 18044 cycles
    test_vector_pod_ins_back() takes 1932 cycles
    test_vector_pod_ins_back() takes 2000 cycles
     test_deque_str_ins_back() takes 91116 cycles
     test_deque_str_ins_back() takes 90132 cycles
      test_list_str_ins_back() takes 106108 cycles
      test_list_str_ins_back() takes 106116 cycles
    test_vector_str_ins_back() takes 88660 cycles
    test_vector_str_ins_back() takes 89152 cycles

(Test results on Windows are similar, though results with MSVC w/o
STLport is too slow to be considered optimized.)

So I am right that std::vector is fast only if insertions (and
erasures) occur at the end of the container.  However, I must admit
that std::deque has surprising performance beyond my hope, and
std::list is a little slow.  I suppose I really should use std::deque
more often.

Best regards,

Yongwei


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> >>Well, maybe. It is not about modifying elements of container, but
> >>container itself:
> >>
> >>// invalid code!
> >>for(vector<Foo>::const_iterator x = v.begin(); x != v.end(); ++x)
> >>     if(const_condition(x))
> >>         v.push_back(some_value);
> >
> >
> > It's hard to look at this as anything but sloppy code. I agree with
>
> Well, the code was presented to showcase the fact that the problem has a
> little to do with const or non-const iterator.

As I understand it, you've been arguing that iterating with indices is
better than with iterators because of the possibility of modifying the
vector. Unintentionally modifying a vector while iterating over it is
what I think is sloppy.

> > Attila and others, it's not unreasonable to expect the programmer to
> > know what he's doing.
>
> The code written exactly as it is here is indeed rare.
>
> The thing is that many classes that use vector in implementation have to
> store 'positions' in the vector. Now thing to decide is whether those
> positions are to be stored as iterators or indicies.

Indices are often the better choice. One advantage of iterators is that
they make it easier to change your mind about what kind of container
you're using. If you're using indices everywhere and decide you want to
change to a list, you've got more work to do. If you're using
iterators, you may get away with just changing a typedef.

> IME it is not a
> good idea to store vector iterators into non-local variables for exactly
> this reason (that is the actual point). BTW, that does not mean you can
> be totaly careless about positions stored as you mutate your vector. It
> is just that with indicies things seem to be more straight.

Obviously, if you are mutating a vector, you should know what you are
doing. However, even if the vector is mutating, you can still use
iterators:

for (it = vec.begin(); it != vec.end(); ++it) {

   // ...

   if (SomeCondition(it)) {
      std::vector<type>::size_type index(it - vec.begin());

      // mutate the vector

      it = vec.begin() + index;
   }

   // ...

}

> E.g. our favorite example, you insert the line in the editor, you have
> to check whether some stored line indicies are past the line inserted
> and increment them. With iterators that would mean to obtain offset
> (actually line index) before insert is performed, then insert, then
> recalculate new iterators. IMHO more error-prone and certainly more work
> to do.

Hey, just like my example. ;-) Yes, it is more work and more
error-prone to do it that way; you wouldn't unless there was some
compelling reason (like a significant amount of code in the "// ..."
sections that require iterators).

> > And in any case, debugging standard libraries
> > will catch this error right away.
>
> Actually, this is the problem. Code like this might not do anything
> wrong for months (because iterators get invalidated only when vector's
> internal buffer resizes). This is the worst kind of bugs.

No, I mean that a debugging standard library will literallly catch it
right away; when a vector operation is called that invalidates
iterators, and you try to use an invalid iterator, the debugging
library will let you know. It will catch "the worst kind of bugs".

Bob


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"White Wolf" <wolof@freemail.hu> writes:

| >>> At the extreme, one can consider an integer value as a canonical form
| >>> of iterator -- if you think of a sequence as a mapping from |N to
| >>> <value>, then the domain of the mapping is a set of iterators.  Of
| >>> course, there are various irregulatiries that do not feet nicely in
| >>> that picture, but most well-behaved sequences do.
| >>
| >> It is (IMHO) a simplification, which has no reason to exist, other than
| >> old habbits.
| >
| > "old habbits"?  Please explain.
| 
| Yep.  Thinking in idexes.  Its like when you say: I saw Claudia Schiffer on 
| the teli.

It is like?  Please don't hesitate to explain better, because I really
do not see the connection.

| You have seen electrons hitting and illuminating a special 
| surface behind thick glass.  We man are simple, and it is enough.  But if we 
| start to talk about the CRT itself, we have to realize that it is not the 
| picture: it just creates it.
| 
| Same with iterators and indices.  Iterators are *not* *the* order.  They 
| *provide* *an* order.  But an index (if we both understand the same when we 
| say this term) *is� the order.  It is an ordinal number.

Huh.  That is too dense for me.  Please explain again, preferably
without analogy that does not improve my understanding of what you're
saying. 

[...]

| I really feel silly, but I am unable to explain it better. :-(

-- 
                                                       Gabriel Dos Reis 
                                           gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> Bob Bell wrote:
> > I'll cut to the chase: here's how the word processor I mentioned
> > worked. The text document data structure was not a vector, deque, or
> > list, and std::string was nowhere in sight either. The document was an
> > array of pointers to character buffers. Each buffer was allowed to grow
> > to around 1500 characters; any buffer that grew too large would be
> > split into two buffers. There was no attempt to align buffer endings
> > with line endings; because of word wrap and the fact that the user
> > could change the margins, doing that was pointless. There was also an
> > auxilliary data structure that was essentially an array of pointers
> > that pointed to the beginnings of lines; this data structure would be
>
> Well, that is one of possible implementation. Anyway, the part about
> "array of pointers pointing to the beginnings of lines" sounds quite
> similar to "vector<string>". Not the same thing, but quite close - just
> thing that you would have one buffer per one line and you are there :)

Not really; the array of pointers to line beginnings points to
character buffers owned by something else; because of word wrapping,
line beginnings change quite frequently; having some kind of "container
of lines" data structure would require sloshing characters back and
forth through lines. It's more efficient to just rebuild the line
pointers at the appropriate times.

> Of course, it seems like your implementation had to deal with somewhat
> different requirements, essentially since you mention "word-wrap", it
> was perhaps rather paragraph based rather than lines based.
>
> Anyway, quite equivalent solution here is vector<paragraph>...

Except you don't really want to do it that way, because of the
possibility of a user who creates one giant paragraph for the whole
document. Now inserting a character at the beginning means moving a lot
of characters. That's the point of maxing each buffer out at around
1500 characters (actually, computers are a lot faster today, so today
you could probably bump that max number up a bit); it places a limit on
how many characters you'll ever have to move around on an insert (while
occasionally requiring allocation of a new buffer).

Text editors make it clear that object-oriented decomposition based on
user-perceivable objects is not always the best approach. Just because
there are things the user perceives as "lines" and "paragraphs" does
not mean that the underlying implementation is best specified this way
(I remember a long time ago a guy I knew wanted to specify a text
editor by having a document that was a container of paragraphs, and a
paragraph as a container of words, and a word as a container of
characters; it still makes me shudder).

> > The main reason no std types were used was that this was all done in C;
> > if it were done in C++, I can see std::vector being used. No way would
> > std::string be used.
>
> It depends. If there is no furhter formating involved, string is good
> both for line or paragraph version.

You could use std::string as the character buffer, but for this purpose
there really isn't much advantage of that over a std::vector<char>, and
std::vector has performance guarantees that std::string lacks.

> In my recent wordprocessor engine, capable of full formating (including
> nested tables), it is Vector<Part>, and this Part is quite complicated
> structure that can (if Part is paragraph) contains String which in turn
> contains serialized content of paragraph. It can also contain
> information about table - if that is true, that table information
> contains grid of RichText cells - RichText represents text - nested
> tables obviously need recursive data model.
>
> If you are interested in overviewing this code, you can download it from
> upp.sf.net (download U++, install and go for RichText package. You can
> try it using "UWord" example. And, btw, at the moment it is not
> optimized for extremly long texts, but that really has nothing to do
> with containers used, just time constraints for development - I am not
> speking about simple lines editor. But than again, you can try editor
> based on Vector<String> as described above in main TheIDE window...).

Thanks, when I get some time, I'll take a look; the structure of text
editors has always been interesting to me.

Bob


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

White Wolf wrote:
> Gabriel Dos Reis wrote:

>>>> At the extreme, one can consider an integer value as a
>>>> canonical form of iterator -- if you think of a sequence as a
>>>> mapping from |N to <value>, then the domain of the mapping is
>>>> a set of iterators.  Of course, there are various
>>>> irregulatiries that do not feet nicely in that picture, but
>>>> most well-behaved sequences do. 

> Iterators are *not* *the* order.  They *provide* *an* order.

Iterators are not the order but neither are indices. A (total) order 
is an *operator*. For iterators i, j into the same sequence, i <= j 
holds exactly when j is reachable from i. Reachability is defined 
through the increment operator. Now, when you want to access the 
sequence by index you need to impose a numbering on its elements that 
will also be defined operationally. This is very visible in walking a 
tree; relating indices to addresses in an array can be seen the same 
way.

You seek the "real" order behind iterators, but going to numbers does 
not take you any deeper. It is just relabeling. In fact, the three 
ordered sets of elements, of indices and of iterators are all 
isomorphic. Of course, the iterator as a code structure "provides" an 
order because it is an OO bundle of allowable things to do with 
indices into that sequence.

> And the "ordinal number" does not even show up in our code.

Members of an ordered set do show up.


Martin

-- 
War doesn't determine who's right but who's left.
--White Wolf

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Gabriel Dos Reis wrote:
> "White Wolf" <wolof@freemail.hu> writes:
[SNIP]
>> Same with iterators and indices.  Iterators are *not* *the* order.  They
>> *provide* *an* order.  But an index (if we both understand the same
>> when we say this term) *is� the order.  It is an ordinal number.
>
> Huh.  That is too dense for me.  Please explain again, preferably
> without analogy that does not improve my understanding of what you're
> saying.
>
> [...]
>
>> I really feel silly, but I am unable to explain it better. :-(

Have you missed the line above?  If anyone has better English skills to 
depict the difference between an ordinal number/order and an iterator: all 
help is welcome.

-- 
WW aka Attila
:::
Wear short sleeves! Support your right to bare arms! 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Martin Eisenberg wrote:
> White Wolf wrote:
[SNIP]
>> Iterators are *not* *the* order.  They *provide* *an* order.
>
> Iterators are not the order but neither are indices.
[SNIP]

One more try, before i give up.  And index is an ordinal number.  Ordinal. 
Running from 0-N-1, where N is the number of the elements in the sequence 
(adressable by the index).  So therefore indices, as they are ordinal 
numbers, *are* the order itself.  It does not matter, that we (possibly) 
reach the real elements of our sequence via an array of pointers 
(sorted/arramged by come criteria), the index is the order, in which those 
pointers are arranged.  So indices are the order, as they are an ordinal 
number.

Iterators however (their most generic form, random access ones are 
different) are nto the order itself.  They implement the order, by allowing 
ordered traversal.  IMHO this is quite a bit of difference.  Of course, one 
could say that the iterator begin() corresponds to 0 and the end() to N. 
Which is true, but it is a correspondence, that iterator is not the order, 
it only procides it for us but its traversal mechanisms.  Of course, with 
random access iterators the difference between indices and iterators is not 
so visible, but, IMHO, it is with other ones.

>> And the "ordinal number" does not even show up in our code.
>
> Members of an ordered set do show up.

I don't get this.  Sorry.

-- 
WW aka Attila
:::
The act of giving is more important than the merit of the receiver. 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Dietmar Kuehl wrote:
[SNIP]
> - The algorithms library is quite incomplete although most of the basic
>   algorithms are all there. It is just a lack of some need combinations
>   or a lack of flexibility. For example, to use 'mismatch()' you need
>   to know which of the two sequences is shorter and pass it in as first
>   sequence. There is no overload taking two pairs of iterators or two
>   ranges which makes it effectively impossible to use 'mismatch()' on
>   two input sequences which are quite frequent in my code because it is
>   often trivial to create an input iterator but not that easy to create
>   even a forward iterator. Another example is the lack of something
>   like 'copy_until()'. Of course, a 'find_if()' followed by a 'copy()'
>   on the subsequence does the trick - except that it again does not
>   cover the input iterator case.

For the above, copy_until, you probably want to take a look at my earlier 
post (direct reply to Scott), where I describe the "canonical" form of 
iteration from another language, where filtering (FOR) and escaping the loop 
(WHILE, it could be UNTIL) is part of *all* iterating commands (COPY, ERASE 
etc.) and even (with pater dabatase drivers) of the indexing command, to be 
able to get an orderd but also limited view of the records.  I think that 
system is very good.  You do not need to make special iterators, but the 
"inner loop" just takes care of the natural FOR (filtering) and WHILE 
(escaping) little functors.  And whan I say functors, I mean functors.  That 
language supperts something very close to functors, called codeblocks. 
These are little functions (which you can pass around as data), they can 
take arguments, they have return values, there is a special iff() function 
to be able to do conditionals inside etc.  I have no idea if these are lamba 
expressions, because I do not know what definitions makes them to be 
lambdas.

I - many times - miss that very straihgtforward, readable and usable form of 
iteration.

> - For many typical cases the algorithms are relatively hard to use due
>   to their need for pairs of iterators. Although the flexibility to
>   operate on subsequences is essential for a fundamental layer, it
>   makes their quite hard and unnecessarily wordy. It should be
>   possible to pass a "range" which exposes a 'begin()' and an 'end()'
>   (i.e. containers happen to be range objects) in addition to overloads
>   taking sequences. Although these algorithms would mere extract the
>   sequence and forward it to a corresponding algorithm, this has a huge
>   impact on usability and also allows algorithm chaining, i.e. passing
>   the result of one algorithm immediately as the input of another
>   algorithm.

I completely agree.  I *think* if Francis would read this, he would agree as 
well, from teaching POV.  In that case, it is much-much better to be able to 
introduce algorithms first on full containers, then to need to explain 
iterators first.  I mean if I want to teach my students generic algorithms, 
I do not want the knowledge of iterators to be a prerequisite.  I *know* 
that for professional level teaching this is mandatory, but in case of 
teaching (for example) programming *using* C++ (and not teaching C++ itself) 
one can go a lot without introducing iterators and iterations.  (Of course 
that lot does not include find() functionality, but that could probably be 
intriduced first using strings and npos, because strings are close to our 
mind, and then for the rest and with iterators - as the generic way.)

> - The STL algorithms erroneously operate on a concept combining access
>   and traversal.
[SNIP]

Do you think that you could post us a little before/after code to show the 
meaning and the benefits of this?  I am sure you have a "canonical example" 
somewhere up your sleeves. ;-)

-- 
WW aka Attila
:::
My opinion is neither copyrighted nor trademarked, and it's price
competitive. If you like, I'll trade for one of yours. 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

White Wolf wrote:
>> Just a note, though: Iterators for 'std::deque<..>' are even worse than
>> iterators for 'std::vector<..>' because they get invalidated with every
>> insert (see 23.2.2.3, lib.deque.modifiers, paragraph 1).
>
> Sure.  But we talked about push_back in this case.

My mistake.  As Dietmar pointed out again deque also invalidates
iterators.

Well, all I can say is what I used deque for (so far) was what it is meant
to be used for.  A double ended que.  I am that kind of irrational,
unreasonable and rather special being, that I do not use an ordered
container for lookup, that I do not use a hash table for ordering, and I do
not keep iterators into a que.  It just never occurs to me.

As to my defense, if I do use deque (which is pretty rare) I do read its
documentation.  And I assess the *facts*, and decide what it was made for,
and if that is what I need.  And if I want to insert into it *anywhere* I
check the documentation *myself*.  I never needed to do that.  May that be,
because I do not like hacking crap together, so if I need something else
than a double ended que, than I do not chose a double ended que and then
complain and rant.

And I do not start a riot to ban iterators, because it happens that I 
don't have the brains to understand how they work.  But as I have said: 
I am an expcetional jerk.  Herb will write his next book about me: The 
Exceptional C++ Jerk.

-- 
WW aka Attila
:::
To know recursion, you must first know recursion.



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Bob Bell wrote:
> (I remember a long time ago a guy I knew wanted to specify a text
> editor by having a document that was a container of paragraphs, and a
> paragraph as a container of words, and a word as a container of
> characters; it still makes me shudder).

I do know of one editor that implements 2/3 of that -- a document
is a container of paragraphs, and a paragraph is a container of
characters. For the most part, this is only visible to the user when
applying formatting (characteristics like indentation and line
spacing belong to the paragraph, while characteristics like font
and color belong to the character). There's no such thing as a
"word" except during spell-check.

On the other hand, Microsoft Word DOES make some people shudder...
despite this, it does seem to have enjoyed some small degree of success.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> Except you don't really want to do it that way, because of the
> possibility of a user who creates one giant paragraph for the whole
> document.

Damned users :)

> Now inserting a character at the beginning means moving a lot
> of characters. That's the point of maxing each buffer out at around
> 1500 characters (actually, computers are a lot faster today, so today
> you could probably bump that max number up a bit); it places a limit on

Well, I think that something close to 100KB of 16bit characters is a 
reasonable limit for the 1Ghz machine.

> how many characters you'll ever have to move around on an insert (while
> occasionally requiring allocation of a new buffer).

Hm, do you know how my suggested Paragraph looks like? :) Actually, I 
would try to keep it as simple as possible before somebody complains 
about speed (no premature optimization), but then I see some quite easy 
possible optimizations that can be applied along vector<Paragraph> 
implementation. First of them is to keep line offsets, second is to 
implement it using chunks of 1500 characters max :)

> Text editors make it clear that object-oriented decomposition based on
> user-perceivable objects is not always the best approach. Just because

Agree.

>>It depends. If there is no furhter formating involved, string is good
>>both for line or paragraph version.
> 
> 
> You could use std::string as the character buffer, but for this purpose
> there really isn't much advantage of that over a std::vector<char>, and
> std::vector has performance guarantees that std::string lacks.

Agree. But this is the fault of the standard, not my design.

My String has performance guarantees that are needed here (one of 
reasons why it exists). Actually, when I was speaking about my code, I 
was speaking about Vector<String>, Array<Part> etc...

> Thanks, when I get some time, I'll take a look; the structure of text
> editors has always been interesting to me.

Then we share a hobby. I have always liked to implement them, I must 
have done at least 15 of them since 1985 including some nice code in Z80 
assembly and including one retail box of MS-DOS era that was sold and 
quite popular in my country... (oh, those nice years of closed source 
software when somebody could have been actually paid for hobby 
programming :)

BTW, it took me 2 years and 3 attempts to get the nested tables engine 
right. Now I feel like a full man :)

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

kevin.hall@motioneng.com wrote:
[SNIP]
> I find many algorithms tremendously useful: stable_sort, equal_range,
> etc....  However, I find that the cost of creating a functor far away
> from the place it is actually used is almost always a detriment to
> readability.
[SNIP]

As far as I recall this is being addressed (seriously) by the committee. 
The last info I recall was that at first sight, there seems to be no problem 
in just allowing function-local classes to have external linkage, and 
therefore being useable in algorithm calls.

Please guys and girls of the Committee, if I remember wrong, correct me!

-- 
WW aka Attila
:::
To steal ideas from one person is plagiarism; to steal from many is
research. -- Steven Wright 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> As I understand it, you've been arguing that iterating with indices is
> better than with iterators because of the possibility of modifying the
> vector. Unintentionally modifying a vector while iterating over it is
> what I think is sloppy.

See the thread. I was just reacting to the suggestion that 
const_iterator solves the problem.

>>IME it is not a
>>good idea to store vector iterators into non-local variables for exactly
>>this reason (that is the actual point). BTW, that does not mean you can
>>be totaly careless about positions stored as you mutate your vector. It
>>is just that with indicies things seem to be more straight.
> 
> 
> Obviously, if you are mutating a vector, you should know what you are
> doing. However, even if the vector is mutating, you can still use
> iterators:
> 
> for (it = vec.begin(); it != vec.end(); ++it) {
> 
>    // ...
> 
>    if (SomeCondition(it)) {
>       std::vector<type>::size_type index(it - vec.begin());
> 
>       // mutate the vector
> 
>       it = vec.begin() + index;
>    }
> 
>    // ...
> 
> }

Have you really read my post? Forget about single loop, thing about 
implementation of the whole class. Single loop is just first-line 
explanation of the problem.

Anyway, perhaps you suggest that this is to be used in each single 
method that is going to mutate our editor. OK, but keep in mind there 
will likely be more than single position to the vector stored. 
Complexity will multiply - and all that just because you want to be cool 
and use iterators. There is no real gain for doing so. Replacing with 
list is imaginary, as you need random access anyway and (in other cases) 
vector-like container usually has a performance edge.

Then again, another issue to consider is how you are going to represent 
lines at interface level. Do you plan to have something like

class Editor {
public:
	string GetLine(editor_iterator x);
};

with all invalidation problems?

>>Actually, this is the problem. Code like this might not do anything
>>wrong for months (because iterators get invalidated only when vector's
>>internal buffer resizes). This is the worst kind of bugs.
> 
> 
> No, I mean that a debugging standard library will literallly catch it
> right away; when a vector operation is called that invalidates
> iterators, and you try to use an invalid iterator, the debugging
> library will let you know. It will catch "the worst kind of bugs".

Actually, no:

----
23.2.4.3 - vector modifiers [lib.vector.modifiers]

iterator insert(iterator position, const T& x);
void     insert(iterator position, size_type n, const T& x);
template <class InputIterator>
   void insert(iterator position, InputIterator first, InputIterator last);

-1- Notes: Causes reallocation if the new size is greater than the old 
capacity. If no reallocation happens, all the iterators and references 
before the insertion point remain valid. If an exception is thrown other 
than by the copy constructor or assignment operator of T there are no 
effects.
----

As you see, if capacity is large enough, yout iterators remains valid. 
If your debugging STL library is standard compliant, it cannot catch 
this bug.

Stupid, but standard. (BTW, in my petty library, all mutating operations 
are considered invalidating regardless of capacity issues, exactly for 
this reason).

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Branimir Maksimovic wrote:
> White Wolf wrote:
>>
>> One cannot learn new tricks under the pressure of commercial software
>> development.  The pace is so high, that it is not possible.
>
> One cannot keep up with pace if doesn't learn new tricks.
> That's why we participate in usenet discussions and buy new books.:)

And I also participate in usenet discussions, to make an S out of myself. 
At least history proves that point all too well. :-)

>  And
>> (unfortunately) many shops just run their people until they burn out,
>> then hire freash ones, who has been thaught of the new stuff on
>> university.
>
> I guess such shops does not last for too long.
> Novice programmers are not experienced, and must at least have some
> period to learn something from experienced programmers.

It works on the way that older ones are given the new ones to teach, being 
told that they have a new guy to help them with the loads of work.  Then, 
when the new guys can work on his own (finally, after increasing the old 
ones worktime for weeks to the unbearable) the old one is laid off.  This is 
not an urban legend.  I have a friend in Hungary (he was a colleauge of 
mine), who is over 40 now, and keeps his sources encrypted and only gives 
away binaries.  Yeah, he is probably a bit overcautious (knowing that the 
owner of the shop is an old friend), but... well, he *has* his reasons.

-- 
WW aka Attila
:::
Usenet is like a herd of performing elephants with diarrhea - massive,
difficult to redirect, awe-inspiring, entertaining, and a source of mind -
boggling amounts of excrement when you least expect it. - Gene
Spafford,1992 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> I hope that it will to be too rude and too off-topic to ask you for
> reading about what I really do with memcpy and non-PODs (my last post to
> you). I have spent a lot of time creating that detailed post :)

What do you want me to do? To point out again that I don't care about
what people think C++ should be like if they cannot back up their claims
by the standard? ...or that the pointer of a derived class my by changed
when converting it to a base class using an internal, invisible pointer?
.... or that a system may setup its memory management unit to trap accesses
to padding in non-POD objects?

Actually, I think I have seen a response you made where you noted that
your "well thought out" restriction might turn out to be incomplete and
that you might need to ban inheritance which you did not so far. Right,
you need to ban inheritance, constructor and destructor and a few other
things, too. Then you will end up at PODs and we are in agreement about
the ability to move them using 'memcpy()'.
>
> Just for the record, I do not claim that you can portably move non-PODs.

Even if the claim that you can portably move a single non-POD class is
not backed up by the standard and thus false. I think you claim you can
move certain non-PODs.
-- 
<mailto:dietmar_kuehl@yahoo.com> <http://www.dietmar-kuehl.de/>
<http://www.eai-systems.com> - Efficient Artificial Intelligence

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Carlos Moreno wrote:
> Geoff Carlton wrote:
>
>
>>>>count_if (names.begin(), names.end(), longer_than("Carlos"));
>>>>[...]
>>
>>Even this simple example isn't particularly easy to understand.  Is it
>>counting instances where n.length > "Carlos", or where "Carlos" >
>>n.length?
>
>
> Oh my!!  You don't find "longer than 'Carlos'" easy to understand??
> In fact, a quick glance of the whole line reads like "count names longer
> than Carlos"... In the same oprder (replacing count_if with count, and
> the whole names.begin(), names.end() with "names")  If you really really
> think the above is not "as obvious as it gets", then I think you are in
> far more serious trouble than using or refusing STL algorithms...
>
> Carlos

I grant that the most intuitive explanation is arg.length>name.length,
but I don't really know for sure, and if I were debugging I'd want to
verify that first off.  I feel this points to one of the disadvantages
of trivial functors, which is unnecessary code separation.  So IMO,
obvious as it gets is the actual > right there where you can see it.

Having said that, I can see the attraction for using functors for
anything beyond the most trivial case.

Cheers,
Geoff


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

wij@seed.net.tw wrote:
> k...@gabi-soft.fr wrote:
> >Something called software engineering.  Writing code that can
> >easily be understood and maintained by others.
>
> I am focusing on the question: "But what's the lot part of
> programming I missed by not using STL?"
>
> Your software engineering issue may be a plausible candidate.
> Given plain code fragmenet:
>
>  const size_t SLEN(5);
>  char str1[SLEN],str2[SLEN];
>
>  // copy
>  for(int i=0; i<SLEN; ++i) {
>    str2[i]=str1[i];
>  }
>
>  // qsort
>  myQSort(str1,SLEN);

Apart from a trivial syntax, I failed to see any C++ code.  What I
would have written:

vector<char> str1, str2;
....
str2 = str1;
sort(str1.begin(), str1.end());

If you could not see that the above code is far superior, more elegant,
and easier to be understood, I would not argue with you.

> Should't it be expected maintained by any programmer?
> If it could cause maintainence problem because of not being
> STL based, then what am I expected to say or think?
> On the contrary, insisting on STL (for all programs) can
> cause numerous problems (owing from its usage requirements),
> not to mention the engineering issue which concerned you.
> Exotic programs can be annoying, normally from bad programs,
> hence is not in the consideration, and it could be using
> STL, too.
>
> BTW, STL also belongs to the set of "exotic programs",
> just that specific exotics was made Standard Template Library.
> Should we support? Yes. But the reality makes supporting it
> expensive.

If you want to write only C, that's OK for you (and your colleagues
that can live with that).  But remember that C is simply *not* C++.

Best regards,

Yongwei


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:
> Have you really read my post? Forget about single loop, thing about
> implementation of the whole class. Single loop is just first-line
> explanation of the problem.

I guess the main point is not to have some cast-in-stone rule. "Always
use iterators" is equally as bad as "always use indices."

> Then again, another issue to consider is how you are going to represent
> lines at interface level. Do you plan to have something like
>
> class Editor {
> public:
>       string GetLine(editor_iterator x);
> };
>
> with all invalidation problems?

No, actually I don't; If I had a "getLine()" function of some sort, it
would probably take a line number as an argument.

> >>Actually, this is the problem. Code like this might not do anything
> >>wrong for months (because iterators get invalidated only when vector's
> >>internal buffer resizes). This is the worst kind of bugs.
> >
> >
> > No, I mean that a debugging standard library will literallly catch it
> > right away; when a vector operation is called that invalidates
> > iterators, and you try to use an invalid iterator, the debugging
> > library will let you know. It will catch "the worst kind of bugs".
>
> Actually, no:
>
> ----
> 23.2.4.3 - vector modifiers [lib.vector.modifiers]
>
> iterator insert(iterator position, const T& x);
> void     insert(iterator position, size_type n, const T& x);
> template <class InputIterator>
>    void insert(iterator position, InputIterator first, InputIterator last);
>
> -1- Notes: Causes reallocation if the new size is greater than the old
> capacity. If no reallocation happens, all the iterators and references
> before the insertion point remain valid. If an exception is thrown other
> than by the copy constructor or assignment operator of T there are no
> effects.
> ----
>
> As you see, if capacity is large enough, yout iterators remains valid.
> If your debugging STL library is standard compliant, it cannot catch
> this bug.
>
> Stupid, but standard. (BTW, in my petty library, all mutating operations
> are considered invalidating regardless of capacity issues, exactly for
> this reason).

I don't think there's anything that stops a debugging standard library
from being more conservative (e.g., invalidating iterators on every
mutating operation); I believe stlport operates this way.

Bob


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

The main problem with standard algorithms, and in particular for_each, is
the syntax.

Defining even the simplest predicate class requires a lot more code than it
seems worth. On top of this, you can't define the functor class right there
where you use it--it has to be a global class (for some obscure compiler
reasons). Lambda calculus would be a welcome thing, as long as its syntax is
simple enough (Boost is not at that stage yet).

I still use a lot of algorithms in my code. Sorting is always done either by
std::sort or by the use of sets and maps. Most of simple searching is done
through the Standard Library.

  I tried to use for_each but have given up, because it increases the total
complexity and decreases readability--mainly because of functors--they don't
fit nicely into the application architecture.

Architectural considerations are important too. Where are you supposed to
keep your functor classes? What should their scope be? I haven't solved that
problem to my satisfaction.

Readability and maintenance are the main things I care about when writing
large applications (200 klocs is my current range).



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"White Wolf" <wolof@freemail.hu> writes:

| Martin Eisenberg wrote:
| > White Wolf wrote:
| [SNIP]
| >> Iterators are *not* *the* order.  They *provide* *an* order.
| >
| > Iterators are not the order but neither are indices.
| [SNIP]
|
| One more try, before i give up.  And index is an ordinal number.  Ordinal.

And a cardinal too.  Why that does not count?

| Running from 0-N-1, where N is the number of the elements in the sequence
| (adressable by the index).

Consider integers as defined by Peano axiom, 0 and succ, as the
basis.  Ignore ordinal and whatever for the moment.  Reevaluate your
assertion.

|  So therefore indices, as they are ordinal
| numbers, *are* the order itself.

Not by any mathematical definition of integers and order I know of.
Do you have refrence on that?

-- 
                                                        Gabriel Dos Reis
                                            gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"White Wolf" <wolof@freemail.hu> writes:

| Gabriel Dos Reis wrote:
| > "White Wolf" <wolof@freemail.hu> writes:
| [SNIP]
| >> Same with iterators and indices.  Iterators are *not* *the* order.  They
| >> *provide* *an* order.  But an index (if we both understand the same
| >> when we say this term) *is� the order.  It is an ordinal number.
| >
| > Huh.  That is too dense for me.  Please explain again, preferably
| > without analogy that does not improve my understanding of what you're
| > saying.
| >
| > [...]
| >
| >> I really feel silly, but I am unable to explain it better. :-(
|
| Have you missed the line above?

If I did, I would not have left it in my message.  I guess.

| If anyone has better English skills to
| depict the difference between an ordinal number/order and an iterator: all
| help is welcome.

You made an assertion incomprehensible to me, that had miraculuously
disappeared under the label "[SNIP]" followed by another assertion
even more incomprehensible to me and apparently unrelated to what I
previously said about integers and iterators.  I'm skeptical that it
is just a matter of English.  I maintain that you have to explain "why
they [indices] are 'old habbits', what the 'new habbits' are, what are
the essential differences and the advantages of the "new habbits" over
the 'old habbits'."  And I recall the references here

     http://www.stepanovpapers.com/


Please no more Claudia Shiffer.

-- 
                                                        Gabriel Dos Reis
                                            gdr@integrable-solutions.net

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

>>Have you really read my post? Forget about single loop, thing about
>>implementation of the whole class. Single loop is just first-line
>>explanation of the problem.
> 
> 
> I guess the main point is not to have some cast-in-stone rule. "Always
> use iterators" is equally as bad as "always use indices."

I can agree on that. I just "mostly use indicies".

>>Actually, no:
>>
>>----
>>23.2.4.3 - vector modifiers [lib.vector.modifiers]
>>
>>iterator insert(iterator position, const T& x);
>>void     insert(iterator position, size_type n, const T& x);
>>template <class InputIterator>
>>   void insert(iterator position, InputIterator first, InputIterator last);
>>
>>-1- Notes: Causes reallocation if the new size is greater than the old
>>capacity. If no reallocation happens, all the iterators and references
>>before the insertion point remain valid. If an exception is thrown other
>>than by the copy constructor or assignment operator of T there are no
>>effects.
>>----
>>
>>As you see, if capacity is large enough, yout iterators remains valid.
>>If your debugging STL library is standard compliant, it cannot catch
>>this bug.
>>
>>Stupid, but standard. (BTW, in my petty library, all mutating operations
>>are considered invalidating regardless of capacity issues, exactly for
>>this reason).
> 
> 
> I don't think there's anything that stops a debugging standard library
> from being more conservative (e.g., invalidating iterators on every
> mutating operation); I believe stlport operates this way.

I see guarantee that under some circumstances mutating operations do not 
invalidate iterators. And there exists code that exploits this. Library 
cannot be more conservative here.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Carlos Moreno wrote:
> Geoff Carlton wrote:
> 
> 
>>>>count_if (names.begin(), names.end(), longer_than("Carlos"));
>>>>[...]
>>
>>Even this simple example isn't particularly easy to understand.  Is it
>>counting instances where n.length > "Carlos", or where "Carlos" >
>>n.length?
> 
> 
> Oh my!!  You don't find "longer than 'Carlos'" easy to understand??

Well, my first guess would be that it compares how tall are persons with 
given names. E.g. fetches data from some global map or SQL database base 
using 'name' as key. Why would you pass string as parameter otherwise?

Actually, this is was one of problems when I tried to use algorithms 
with functors - finding good descriptive name for operation is often 
quite problematic. And example given here is still quite simple.

> think the above is not "as obvious as it gets", then I think you are in
> far more serious trouble than using or refusing STL algorithms...

Well, I would find almost obvious

count_if (names.begin(), names.end(), size_is_greater(strlen("Carlos")));

but in my eyes it is still inferior to simple loop in terms of clarity.

I consider your design somewhat broken in any case, sorry. Passing 
string just to get it compared for length in functor, not a good idea.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

"Geoff Carlton" <clcppm-poster@this.is.invalid> skrev i meddelandet 
news:42f00401$0$11930$5a62ac22@per-qv1-newsreader-01.iinet.net.au...
>
> Carlos Moreno wrote:
>> Geoff Carlton wrote:
>>
>>
>>>>>count_if (names.begin(), names.end(), longer_than("Carlos"));
>>>>>[...]
>>>
>>>Even this simple example isn't particularly easy to understand.  Is 
>>>it
>>>counting instances where n.length > "Carlos", or where "Carlos" >
>>>n.length?
>>
>>
>> Oh my!!  You don't find "longer than 'Carlos'" easy to understand??
>> In fact, a quick glance of the whole line reads like "count names 
>> longer
>> than Carlos"... In the same oprder (replacing count_if with count, 
>> and
>> the whole names.begin(), names.end() with "names")  If you really 
>> really
>> think the above is not "as obvious as it gets", then I think you are 
>> in
>> far more serious trouble than using or refusing STL algorithms...
>>
>> Carlos
>
> I grant that the most intuitive explanation is arg.length>name.length,
> but I don't really know for sure, and if I were debugging I'd want to
> verify that first off.  I feel this points to one of the disadvantages
> of trivial functors, which is unnecessary code separation.  So IMO,
> obvious as it gets is the actual > right there where you can see it.

Ok, so perhaps you use the wrong tools?

In VS2005 you just have to click on the functor name to have it pop up 
in the Code Definition Window. Is this really a language issue?

Perhaps locality means "a mouse click away", and not "on the next couple 
of lines"  ?

>
> Having said that, I can see the attraction for using functors for
> anything beyond the most trivial case.
>

So, with the right tools we might use more functors?


Bo Persson



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

>>Just for the record, I do not claim that you can portably move non-PODs.
> 
> 
> Even if the claim that you can portably move a single non-POD class is
> not backed up by the standard and thus false. I think you claim you can
> move certain non-PODs.

Certain non-PODs on *certain platforms* per *ABI specification*. That is 
  light years away from your interpretation. Some STL implementation do 
this already too (but only for standard types). I guess yasli::vector 
does this also (Andrei speaks about "guarded platform assumptions"), but 
I am unable to pick the code to check it out as the link on Andrei's 
webpage seems to be broken.

Also, all that debate started about iterators - all that I was claiming 
is that under certain circumstances, iterators do not provide enough 
information to implement effective algorithm or even any algorithm at 
all. And my actual problem with iterators is not about this non-POD 
issue, but with 100% standard compliant code.

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler <cxl@volny.cz> wrote:
> [...]
>
> Well, I would find almost obvious
>
> count_if (names.begin(), names.end(), size_is_greater(strlen("Carlos")));
>
> but in my eyes it is still inferior to simple loop in terms of clarity.

  Is it? I'm not a native speaker, but to me this
  seems to be almost plain English.

> I consider your design somewhat broken in any case, sorry. Passing
> string just to get it compared for length in functor, not a good idea.

  He has agreed to that and it's beside the point.

> Mirek

  Schobi

-- 
SpamTrap@gmx.de is never read
I'm Schobi at suespammers dot org

"Coming back to where you started is not the same as never leaving"
 Terry Pratchett



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

In article <3lbmg6F11r4luU1@individual.net>, Mirek Fidler <cxl@volny.cz> 
writes
>Actually, this is was one of problems when I tried to use algorithms
>with functors - finding good descriptive name for operation is often
>quite problematic. And example given here is still quite simple.

Actually good names are impossible unless you take the context of use 
into consideration. Without that context your names have to do too much.

-- 
Francis Glassborow      ACCU
Author of 'You Can Do It!' see http://www.spellen.org/youcandoit
For project ideas and contributions: http://www.spellen.org/youcandoit/projects


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Mirek Fidler wrote:

> Well, I would find almost obvious
> 
> count_if (names.begin(), names.end(), size_is_greater(strlen("Carlos")));
> 
> but in my eyes it is still inferior to simple loop in terms of clarity.
> 
> I consider your design somewhat broken in any case, sorry. Passing 
> string just to get it compared for length in functor, not a good idea.

I thought we had agreed on storing the length in the function object:

class longer_than
{
     size_t len;
public:
     longer_than (const string & s) : len(s.length()) {}

     // ...
};


Carlos
--

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

> I thought we had agreed on storing the length in the function object:
> 
> class longer_than
> {
>      size_t len;
> public:
>      longer_than (const string & s) : len(s.length()) {}
> 
>      // ...
> };

Yes, but this time I consider wrong to pass string parameter to the 
constructor :)

Mirek

      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Gabriel Dos Reis wrote:
> "White Wolf" <wolof@freemail.hu> writes:
>
>> Martin Eisenberg wrote:
>>> White Wolf wrote:
>> [SNIP]
>>>> Iterators are *not* *the* order.  They *provide* *an* order.
>>>
>>> Iterators are not the order but neither are indices.
>> [SNIP]
>>
>> One more try, before i give up.  And index is an ordinal number.
>> Ordinal.
>
> And a cardinal too.  Why that does not count?

Because I am not church going.

>> Running from 0-N-1, where N is the number of the elements in the
>> sequence (adressable by the index).
>
> Consider integers as defined by Peano axiom, 0 and succ, as the
> basis.  Ignore ordinal and whatever for the moment.  Reevaluate your
> assertion.

I am not ignoring my argument, thank you.

>>  So therefore indices, as they are ordinal
>> numbers, *are* the order itself.
>
> Not by any mathematical definition of integers and order I know of.
> Do you have refrence on that?

I am sorry, but I use my own thoughts and I am reluctant of shifting this 
topic into a war of "who can list more authorities".  As I have said in my 
earlier post: if you care about what I say, please try to understand what 
*I* say.  If you don't care, please do not care to post sarcastic remarks 
either.  Thank you.


-- 
WW aka Attila
:::
Live simply so that others may simply live... - Gandhi 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Gabriel Dos Reis wrote:
[SNIP]
> You made an assertion incomprehensible to me, that had miraculuously
> disappeared under the label "[SNIP]" followed by another assertion
> even more incomprehensible to me and apparently unrelated to what I
> previously said about integers and iterators.  I'm skeptical that it
> is just a matter of English.  I maintain that you have to explain "why
> they [indices] are 'old habbits', what the 'new habbits' are, what are
> the essential differences and the advantages of the "new habbits" over
> the 'old habbits'."  And I recall the references here
>
>     http://www.stepanovpapers.com/
>
>
> Please no more Claudia Shiffer.

What is the attitude for?  I have explained what *I* meant, to the best of 
my abilities.  if you would like to know something else, not what I meant 
(which is a quite understandable intention, most people have it who ever 
talked to me) please feel free to ask someone else for his/her thoughts.

I am sorry that you have not understood what I have written.  It surely is 
my fault.  But no matter how harsh or sarcastic words you use in your 
pressure to make me say something else, there is nothing else I can or want 
to say.

-- 
WW aka Attila
:::
The facts expressed here belong to everybody, the opinions to me. The
distinction is yours to draw... 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Bartosz Milewski wrote:
> Architectural considerations are important too. Where are you supposed to
> keep your functor classes? What should their scope be? I haven't solved
> that problem to my satisfaction.

I keep them inside the class, for example for different orderings.  If it is 
something, which does not really belong to the "oen for all" interface 
(dependencies etc.) I keep them as globals (in some namespace, usually not 
unnamed) inside an implementation file, where they are used.

As I have said earlier, the function-local-classes-to-have-external-linkage 
topic is seriously considered.  As I recall it has not been left out earlier 
for any obscure reason, but for a very specific one: committe time.  Simple 
there was no time to see, if they would create any trouble if they had 
external linkage.  Last thing I recall was that some compilers can even do 
it today (I may remember that one wrong, I ususally have terrible headches 
during the meetings), and that nobody could think of anything against it. 
So if all is well, we may see them soon work in many compilers.

-- 
WW aka Attila
:::
A foolproof method for sculpting an elephant: first, get a huge block of
marble, then you chip away everything that doesn't look like an elephant. 



      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

White Wolf wrote:
> Martin Eisenberg wrote:
> > White Wolf wrote:
> [SNIP]
> >> Iterators are *not* *the* order.  They *provide* *an* order.
> >
> > Iterators are not the order but neither are indices.
> [SNIP]
>
> One more try, before i give up.  And index is an ordinal number.  Ordinal.

Nope. index does not necessarily has type of ordinal number in c++.

> Running from 0-N-1, where N is the number of the elements in the sequence
> (adressable by the index).

Wrong. Nothing is addressable by index. index is just argument to
indexing
*function*.

  So therefore indices, as they are ordinal
> numbers, *are* the order itself.

No. indexing function does not have to produce ordered set.

  It does not matter, that we (possibly)
> reach the real elements of our sequence via an array of pointers
> (sorted/arramged by come criteria), the index is the order, in which those
> pointers are arranged.

No, order in which those pointers are arranged depends on indexing
function not arguments of function.

> Iterators however (their most generic form, random access ones are
> different) are nto the order itself.

Iterators are different then usual indexes
 (arguments to an indexing function)
in a way that they are argument of unary  obj_acc_function(iterator)
rather then binary obj_acc_function (container,index).
That means that iterator is directly associated with only one object,
but index don't have to be. f(c1,index) f(c2,index) could yield two
different objects.
In all other respects(regarding order, supported operators and such)
they can be identical. If we pick raw pointers as iterators they can
have same type as integer indexes.

Greetings, Bane.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]

Bob Bell wrote:
> I don't think there's anything that stops a debugging standard library
> from being more conservative (e.g., invalidating iterators on every
> mutating operation); I believe stlport operates this way.

There are some operations that the standard guarantees will NOT
invalidate iterators (i.e. use reserve for capacity, then use
push_back up to that size), and there is some code that relies on
these guarantees. A debugging standard library that invalidates on
every mutating operation would trigger false errors in such code.

A "conservative" debugging library would minimize how often this
happens automatically; i.e. grow only as much as absolutely required
(so that push_back without reserve would grow at every insertion,
invalidating every time).

I don't know about stlport, but I hope it doesn't invalidate all
iterators on push_back when capacity>size.


      [ See http://www.gotw.ca/resources/clcm.htm for info about ]
      [ comp.lang.c++.moderated.    First time posters: Do this! ]