Hi,

Imagine that i have a vector container, which contains a list of items
that need downloading, and i want X amount of these items to download
simultaneously.

These items are being transfered over a custom protocol, not something
standard like http so i cant use a pre made library, rather im dealing
with raw socket operations.

How do i create a decent stable architecture for this purpose?

So my vector is going be the thread safe queue object so to speak,
that my download objects will access, then process the item from the
vector.

Problem is whats the best design approach to do this?

I have been looking at the Boost library for threading, but some
people have said i can just use asynchronous I/O and eliminate the
need for threads, but i have no idea how i would implement asio?

My idea for threading would be to have 1 socket / download object per
thread, and each thread would lock/unlock the vector queue when
required. Would this work? or is it over the top?

I'm really just after a good understanding of the design so i can
implement it as hassle free as possible because threading is scary :p.

This is for linux, so does anyone know if theres any existing code for
this type of design i could use, or look at for ideas, any library's
that exist?

Thanks for any help,

Jason

Jason <jecheney@gmail.com> writes:
> Imagine that i have a vector container, which contains a list of items
> that need downloading, and i want X amount of these items to download
> simultaneously.

[...]

> My idea for threading would be to have 1 socket / download object per
> thread, and each thread would lock/unlock the vector queue when
> required. Would this work?

It should. I would implement this based on a ring buffer with two
counting semaphores (one for the number of free slots and one for the
number of queued download requests) and a mutex lock on the receiving
end (assuming their is one 'producer thread' queueing download
requests and n 'download threads' doing the actual work). Initially,
the 'free slots' semaphore would be set to size of the buffer and the
'queued requests' semaphore to zero. The download threads would then
all block on the 'queued requests' semaphore. The 'producer thread'
would then first do a sem_wait on the 'free slots' semaphore and after
that returns, write a new download request in the 'current' slot (no
furhter interlocking necessary and do a sem_post on the 'queued
requests' semaphore. The download threads would be doing a sem_wait on
that. Everytime this call returns in a download thread, the thread
would acquire the mutex, remove the current 'to-be-consumed' item from
the buffer, unlock the mutex, post the 'free slots' semaphore and
proceed to downloading. After download and download-related processing
have completed, it would again sem_wait on the queued requests
semaphore.