f



socket broken pipe error on sending huge data.

Dear All,
I have developed a simple client server application, the server needs
to send around 1.5 MB of data to the client for each transaction
continously. But the server is only able to send only 81660 bytes of
data and throws
"ERROR writing to socket: Broken pipe"

1) Cannot we send huge data through sockets and that too continously ?
2) What can be possible reason for this issue?
3) How to resolve the same?
4) Any code snippet will be of great use.
I am a NO expert in this area, so all comments are most welcome.
/////// SERVER CODE SNIPPET///
int serverConnection()
{
     int sockfd, newsockfd, portno, clilen;
     char buffer[256];

     struct sockaddr_in serv_addr, cli_addr;

	 int n;

	 sockfd = socket(AF_INET, SOCK_STREAM, 0);

	 if (sockfd < 0)
        error("ERROR opening socket");

	 bzero((char *) &serv_addr, sizeof(serv_addr));

	 portno = 5902;

	 serv_addr.sin_family = AF_INET;
     serv_addr.sin_addr.s_addr = INADDR_ANY;
     serv_addr.sin_port = htons(portno);

	 if (bind(sockfd, (struct sockaddr *) &serv_addr,
              sizeof(serv_addr)) < 0)
              error("ERROR on binding");

	 listen(sockfd,5);
     clilen = sizeof(cli_addr);

	 newsockfd = accept(sockfd,
                 (struct sockaddr *) &cli_addr,
                 &clilen);

	 if (newsockfd < 0)
          error("ERROR on accept");

	 bzero(buffer,256);

	 n = read(newsockfd,buffer,255);

	 if (n < 0) error("ERROR reading from socket");

	 printf("Here is the message: %s\n",buffer);


	 do{
	 fillBuffer
	 n = write(newsockfd, &HUGEDATA,sizeof(HUGEDATA));
	if( n < 1 && errno != EINTR ) {
	error("ERROR writing to socket");
	return 0;
	}

	if( n == EINTR ) {
	fprintf(stdout,"SOAP:\t:write interupted...tyring again");
	continue;
	}


     }while(1);

	 if (n < 0) error("ERROR writing to socket");
     return 0;

}
///////////
////////////////CLIENT SNIPPET///
int receivedData()
{
  int sockfd,  n;
    struct sockaddr_in serv_addr;
    struct hostent *server;

    char buffer[256];

    sockfd = socket(AF_INET, SOCK_STREAM, 0);
    if (sockfd < 0)
        error("ERROR opening socket");
  	server = gethostbyname("localhost");

    if (server == NULL) {
        fprintf(stderr,"ERROR, no such host\n");
        exit(0);
    }

    bzero((char *) &serv_addr, sizeof(serv_addr));
    serv_addr.sin_family = AF_INET;
    bcopy((char *)server->h_addr,
         (char *)&serv_addr.sin_addr.s_addr,
         server->h_length);
    serv_addr.sin_port = htons(5902);
    if (connect(sockfd,&serv_addr,sizeof(serv_addr)) < 0)
        error("ERROR connecting");


	printf("Please enter the message: ");
    bzero(buffer,256);
	strcpy(buffer,"rohit");
    n = write(sockfd,buffer,strlen(buffer));
    if (n < 0)
         error("ERROR writing to socket");
   while(1)
   {
   n = read(sockfd,&HUGEDATA,sizeof(HUGEDATA));
	if (n < 0)
         error("ERROR reading from socket");

  	}

    return 0;
}

0
7/16/2007 5:23:20 PM
comp.unix.programmer 10848 articles. 0 followers. kokososo56 (350) is leader. Post Follow

12 Replies
1149 Views

Similar Articles

[PageSpeed] 9

On Jul 16, 10:23 am, rohit <PowerPCDevelo...@gmail.com> wrote:
> Dear All,
> I have developed a simple client server application, the server needs
> to send around 1.5 MB of data to the client for each transaction
> continously. But the server is only able to send only 81660 bytes of
> data and throws

Short version -- the 'read' needs to be in a 'while' loop. You need to
keep reading until you get all the data. If you stop after you get one
bit of data, the server will soon be sending to nothing.

DS

0
David
7/16/2007 6:51:23 PM
On Mon, 16 Jul 2007 17:23:20 -0000, rohit 
  <PowerPCDeveloper@gmail.com> wrote:
>
>
> Dear All,
> I have developed a simple client server application, the server needs
> to send around 1.5 MB of data to the client for each transaction
> continously. But the server is only able to send only 81660 bytes of
> data and throws
> "ERROR writing to socket: Broken pipe"
>
> 1) Cannot we send huge data through sockets and that too continously ?
> 2) What can be possible reason for this issue?
> 3) How to resolve the same?
> 4) Any code snippet will be of great use.
> I am a NO expert in this area, so all comments are most welcome.
> /////// SERVER CODE SNIPPET///

A broken pipe error means the receiving process died.  Maybe you could 
use strace or some other tool to find out why.


-- 
Using TSO is like kicking a dead whale down the beach.
		-- S. C. Johnson
0
marcumbill (1024)
7/16/2007 7:04:58 PM
rohit wrote:
> Dear All,
> I have developed a simple client server application, the server needs
> to send around 1.5 MB of data to the client for each transaction
> continously. But the server is only able to send only 81660 bytes of
> data and throws
> "ERROR writing to socket: Broken pipe"
> 
> 1) Cannot we send huge data through sockets and that too continously ?
> 2) What can be possible reason for this issue?
> 3) How to resolve the same?
> 4) Any code snippet will be of great use.

In short : You can't assume that writing or reading (large) amounts of 
data in one read() / write() call succeeds.




	Igmar
0
igmar2 (52)
7/17/2007 7:50:47 AM
Thanks, b/w can we use UDP i.e datagrams to send huge amounts (approx
2MB) of data per transaction continously?
Rohit
On Jul 17, 12:50 pm, Igmar Palsenberg <ig...@palsenberg.local> wrote:
> rohit wrote:
> > Dear All,
> > I have developed a simple client server application, the server needs
> > to send around 1.5 MB of data to the client for each transaction
> > continously. But the server is only able to send only 81660 bytes of
> > data and throws
> > "ERROR writing to socket: Broken pipe"
>
> > 1) Cannot we send huge data through sockets and that too continously ?
> > 2) What can be possible reason for this issue?
> > 3) How to resolve the same?
> > 4) Any code snippet will be of great use.
>
> In short : You can't assume that writing or reading (large) amounts of
> data in one read() / write() call succeeds.
>
>         Igmar


0
7/18/2007 3:51:49 AM
rohit wrote:

Please don't top-post.

> On Jul 17, 12:50 pm, Igmar Palsenberg <ig...@palsenberg.local> wrote:
>> rohit wrote:
>>> Dear All,
>>> I have developed a simple client server application, the server needs
>>> to send around 1.5 MB of data to the client for each transaction
>>> continously. But the server is only able to send only 81660 bytes of
>>> data and throws
>>> "ERROR writing to socket: Broken pipe"
>>> 1) Cannot we send huge data through sockets and that too continously ?
>>> 2) What can be possible reason for this issue?
>>> 3) How to resolve the same?
>>> 4) Any code snippet will be of great use.
>> In short : You can't assume that writing or reading (large) amounts of
>> data in one read() / write() call succeeds.
>>
> Thanks, b/w can we use UDP i.e datagrams to send huge amounts (approx
> 2MB) of data per transaction continously?
> Rohit

Not reliably unless you add your own higher layer protocol over UDP.

-- 
Ian Collins.
0
ian-news (10155)
7/18/2007 4:08:18 AM
In article <1184730709.367083.129570@j4g2000prf.googlegroups.com>,
 rohit <PowerPCDeveloper@gmail.com> wrote:

> Thanks, b/w can we use UDP i.e datagrams to send huge amounts (approx
> 2MB) of data per transaction continously?

The maximum size of a UDP datagram is 64K bytes, including the UDP 
header.

-- 
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***
0
barmar (6125)
7/18/2007 4:55:55 AM
On Jul 17, 12:50 am, Igmar Palsenberg <ig...@palsenberg.local> wrote:

> In short : You can't assume that writing or reading (large) amounts of
> data in one read() / write() call succeeds.

Although in this case, it does succeed. It just doesn't read as much
data as he might like.

DS

0
David
7/18/2007 7:33:50 AM
On Jul 18, 12:33 pm, David Schwartz <dav...@webmaster.com> wrote:
> On Jul 17, 12:50 am, Igmar Palsenberg <ig...@palsenberg.local> wrote:
>
> > In short : You can't assume that writing or reading (large) amounts of
> > data in one read() / write() call succeeds.
>
> Although in this case, it does succeed. It just doesn't read as much
> data as he might like.
>
> DS

Please clarify for following: I want to send 1.5 MB of data from
server to client continously
 (a)  So from the server part do i need to create a single buffer of
1.5 MB ? Or make small buffers from the server itself?
(b) Then at client end, do i need to allocate a small buffer area,
lets say 100 KB each, keep on getting the data and add in my 1.5MB
buffer ? Till i read all the data?

As posted in previous post, I am a novice in socket programming area,
so all comments, code snippet, knowledge sharing things are most
welcome.

0
7/19/2007 3:50:54 PM
On Jul 18, 12:33 pm, David Schwartz <dav...@webmaster.com> wrote:
> On Jul 17, 12:50 am, Igmar Palsenberg <ig...@palsenberg.local> wrote:
>
> > In short : You can't assume that writing or reading (large) amounts of
> > data in one read() / write() call succeeds.
>
> Although in this case, it does succeed. It just doesn't read as much
> data as he might like.
>
> DS

Also, how can I know in advance how much amount of data a socket can
read or write ?

0
7/19/2007 3:52:56 PM
rohit <rohit.dhamija@gmail.com> writes:

> On Jul 18, 12:33 pm, David Schwartz <dav...@webmaster.com> wrote:
>> On Jul 17, 12:50 am, Igmar Palsenberg <ig...@palsenberg.local> wrote:
>>
>> > In short : You can't assume that writing or reading (large) amounts of
>> > data in one read() / write() call succeeds.
>>
>> Although in this case, it does succeed. It just doesn't read as much
>> data as he might like.
>>
>> DS
>
> Please clarify for following: I want to send 1.5 MB of data from
> server to client continously

Oh!  What do you do when you've done sent it?  If I were you, I would
send it very slowly.  Since the expected age of the liveable universe
is about 15e9 years, I'd not send more than one bit every 1250 years.


>  (a)  So from the server part do i need to create a single buffer of
> 1.5 MB ? Or make small buffers from the server itself?

Depends where these bits come from.  If you can generate them faster
than you can send them, there's no need for a buffer.  Otherwise, you
may need to generate them beforehand, or you may send them slower.  If
you generate them beforehand, it might be worthwhile to store them
outside of the main memory.  With a good OS, this would be done
automatically,, but if you don't have a true OS, you may have to do it
yourself, loading small chunks from a mass data storage into RAM
buffers, etc.


> (b) Then at client end, do i need to allocate a small buffer area,
> lets say 100 KB each, keep on getting the data and add in my 1.5MB
> buffer ? Till i read all the data?

Depends what you need to do on the other end.  You've only said you
needed to send the data continuously, not that it was to be subjected
to any processing on the other side.  You could just as well throw it
away as soon as received.  Or if you can process it faster than you
can receive it, then you don't need to buffer it.  Or if you have
enough RAM you can just store the whole 1.5MB in RAM.  Or you may also
need to store it on mass storage, then you may need buffers of a size
depending on the relative speed at which you can receive and you can
write to the mass storage.  How can we guess? It's up to you to study
it!

In the mean time, you may be interested in knowing that the IP
protocol cannot send data in packages of more than 64KB (usually it
uses smaller packet size, to accomodate the smaller packet size of the
underlying network layers), therefore even if TCP provide a stream
abstraction, you won't receive more than 64KB (usually much less) "at
once". 

Of course, allocating one big buffer, sending it in one write syscall,
and on the other end allocating one big buffer, and receiving it in
one read syscall MAY work and be simplier to write.  But it's quite
probable that you'll have to implement at least one loop to do that,
because for example, the unix read/write syscall API allows for
partial transfers (see the EINTR error from man 2 read and man 2
write, and their return value).


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
Wanna go outside.
Oh, no! Help! I got outside!
Let me back inside!
0
pjb (7869)
7/19/2007 4:12:27 PM
rohit wrote:
> On Jul 18, 12:33 pm, David Schwartz <dav...@webmaster.com> wrote:
>> On Jul 17, 12:50 am, Igmar Palsenberg <ig...@palsenberg.local> wrote:
>>
>>> In short : You can't assume that writing or reading (large) amounts of
>>> data in one read() / write() call succeeds.
>> Although in this case, it does succeed. It just doesn't read as much
>> data as he might like.
>>
>> DS
> 
> Also, how can I know in advance how much amount of data a socket can
> read or write ?
> 
You shouldn't have to.  Just keep calling write until all of the data is
transferred and read until there is nothing left to read.

-- 
Ian Collins.
0
ian-news (10155)
7/19/2007 8:54:15 PM
On Jul 19, 8:52 am, rohit <rohit.dham...@gmail.com> wrote:

> Also, how can I know in advance how much amount of data a socket can
> read or write ?

You can't because that would be asking the system to predict the
future, which it cannot know.

DS

0
David
7/21/2007 8:04:30 AM
Reply:

Similar Artilces:

Error Code :17410 Error State :null Error Message :No more data to read from socket Stack Trace : java.sql.SQLException: No more data to read from socket
Hi, I see the following errors in the logs: Error Code :17410 Error State :null Error Message :No more data to read from socket Stack Trace : java.sql.SQLException: No more data to read from socket at oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:114) at oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:156) at oracle.jdbc.dbaccess.DBError.check_error(DBError.java:803) at oracle.jdbc.ttc7.MAREngine.unmarshalUB1(MAREngine.java:731) at oracle.jdbc.ttc7.MAREngine.unmarshalSB1(MAREngine.java:690) at oracle.jdbc.ttc7.Oall7.receive(Oall7.java:372) at oracle.jdbc.ttc7...

when using the Matlab "unix" command I get a unix broken pipe error
The file being piped "veryLargeFile.txt " is large > 50000 lines the bash script "test.sh" contains: cat veryLargeFile.txt | grep "DATA0" > data0Lines.txt The Matlab command executed is: unix('test.sh') The following is printed on the command line and the file data0Lines.txt is not created: broken pipe On 01/12/10 8:05 PM, Dan wrote: > The file being piped "veryLargeFile.txt " is large > 50000 lines > the bash script "test.sh" contains: > cat veryLargeFile.txt | grep "DATA0" > data0Lines.tx...

[VxW] Socket Broken pipe Error
Hi All, How to solve the Socket Send Error Broken pipe 32 (0x20) ... Send also crashes after some time when i send large amount of data continuously in vx-works 5.4 i am not using SELECT ...... Obviously I am doing something wrong and would appreciate any help THANKS JITHESH ...

UNIX Socket does not send data until closed
Hi, I am using PHP persistent sockets and have a problem with them. The problem is that PHP only sends the data when the socket is closed. In the code example below the counterparty process (peer) received the connect request from PHP, but then never receives the message sent in the "fwrite" command below - and consequently never generates the reply which should be read by the fgets command. Flushing the socketFP does not make any difference - the only thing that causes the data to be sent is to fclose it (which defeats the purpose of having persistent sockets, which I definitely...

How to I force a Data to be send out at once in socket send
Hi,Every One, I am now writing some application using ACE 5.3 . sorry to disturbing. I have truble in my application which should found network failure as soon as possible. I know,normally tcp send will buffered data in system send buffer and send out later. and ,the ack won't send back immedealy for some function to improve tcp performace such as Neagle .How can i force the send fuction to send the data to the network without buffering in local buffer? What i want to do is when i send out a frame of data via socket,i can get response like n equal to the length of the frame when remo...

How to I force a Data to be send out at once in socket send
Hi,Every One, sorry to disturbing. I have truble in my application which should found network failure as soon as possible. I know,normally tcp send will buffered data in system send buffer and send out later. and ,the ack won't send back immedealy for some function to improve tcp performace such as Neagle .How can i force the send fuction to send the data to the network without buffering in local buffer? What i want to do is when i send out a frame of data via socket,i can get response like n equal to the length of the frame when remote peer received the frame of the data ,or -1 when ...

socket.error: (32, 'Broken pipe'): need help
Hi, I have a simple server-client application with threading. It works fine = when both server and client on the same machine, but I get the following = error message if the server is on another machine: .... ... self.socket.send(outgoingMsg) socket.error: (32, 'Broken pipe') I do not know where to start with?=20 Thanks Junhua When enabling the server, note that you put '' as the IP, and not 'localhost'. if you put an ip: '172.16.1.2', your server will listen on that IP (if it's valid) if you put '', your server will listen on all IPs define...

Unix Stream socket -- "send()" for large amount of data
I'm working on an application where one thread ("producer") will be adding data to a buffer and another ("consumer") will be sending it to a stream socket via "send()" as fast as the socket will take it. Is it better for the consumer to do a single "send()" call each time with all the (un-consumed) data in the buffer? Or is it better to pass no more than some "max_send_size" bytes with each call? (If the system buffer has more than "max_send_size" bytes available, this would mean more calls to send().) From the point of view o...

send-uucp in inn-2.4.0 failing with 'broken pipe' errors
Hi everyone, I recently upgraded the news server installation on my main and secondary news servers to Inn-2.4.0. Since the upgrade, every time 'send-uucp' has been run out of the regular cron job I use to create UUCP news article batche for the handful of member machines which connect via UUCP, it has failed with 'broken pipe' errors. I know the UUCP software is working as mail queues fine, but news batching doesn't seem to be working. Is there anything peculiar to the Inn-2.4.x software which might be causing these problems? Previously the servers were ru...

send more data on the socket > socket buffer can handle
Hello Group, Recently I came across this question asked to me. Is there any flag you can set so that you are allowed to send more data on the socket in the calls to sentto/send, than the internal socket buffer can handle. Is there any way. And if there is no way then what is a way out. Thanks for your help. nagrik In article <1177112559.470678.217590@d57g2000hsg.googlegroups.com>, Vinay Nagrik <jainarunk@gmail.com> wrote: > Hello Group, > > Recently I came across this question asked to me. > > Is there any flag you can set so that you...

Broken pipe error
Hello Group, I am writing a client (browser without the GUI) for a http server. If I serialize all requests before the server is done sending data for all the request then it is OK. For example send 1st request receive part of data of 1 request send 2nd request receive rest of data for both of the requests. However, in the situation if the server has finished sending all data for all pending requests, then my client gets Broken pipe error for example send 1st request receive full data for Ist requst 'Try sending 2nd request' "Receive Broken pipe error. Can someone tell m...

broken pipe error
Hi, occasionaly I find in my apache logs that fastcgi had a broken pipe error with php running as fastcgi. the logs are like this: -- > [Wed Nov 5 18:21:31 2003] [error] [client %ip%] (32)Broken pipe: > FastCGI: comm with server /opt/guide/ppi.searchy.net/cgi-bin/php.fcgi" > aborted: write failed > [Wed Nov 5 18:21:31 2003] [error] [client %ip%] FastCGI: incomplete > headers (0 bytes) received from server > "/opt/guide/ppi.searchy.net/cgi-bin/php.fcgi" -- /opt/guide/ppi.searchy.net/cgi-bin/php.fcgi is the php binary itself. The error occure randoml...

Pipe data through a socket
I have a client/server socket written to perform basic commands. What I am wondering is if I can have the client tell the server.pl tar to stdout back through the socket where the client would gzip from stdin. We the firewall rule sets we have ssh is our only option, but that seems to add more overhead then we like. If this is possible does anyone have an example? Thanks Scott ...

error: error sending
Hi, I hope somebody can help me. For sending mail I configured alpine with the following line: smtp.web.de/tls/user=xxxxxxx@web.de . But every time I tried the message "error sending" occured. To figure out more details I added the parameter debug, but there was no file .pine-debug. After reading the manual I tried: alpine -d 5 and got the message: Argument Error: unknown flag "d", debugging not compiled in Before downloading the source and compile it by my self, I thought I should ask you guys for help. My pine version: alpine-2.00-5.fc11.i586 Best regards Chris...

Web resources about - socket broken pipe error on sending huge data. - comp.unix.programmer

Sending/receiving relationship - Wikipedia, the free encyclopedia
A sending/receiving relationship is one in which a public school district sends some or all of its students to attend the schools of another ...

U.K. Woman Faces 20 Months For Creating Fake Facebook Accounts, Sending Herself Abusive Messages
A woman in Cornwall in the U.K. came up with a creative way to use Facebook to accelerate a family spat, and it earned her 20 months in prison. ...

CHVRCHES - Dear guys, Please stop sending us emails like... - Facebook
Dear guys, Please stop sending us emails like this. This is one of the more polite ones. Other recent classics include "I'm going to give her ...

[...] Facebook Prevents Users From Sending Suspicious Friend Requests To Strangers, Inside Facebook [...] ...
Facebook has begun implementing new security measures to prevent friend request spamming. Some users are being shown a “This Request Can’t Be ...

A better experience for sending Tweets and DMs - Twitter Blog
... panel New Tweet panel in da...... Skip to main content Blog Got an account? Sign in Search form Search &nbsp A better experience for sending ...

How to Go From Working 60 Hours a Week to 40 By Sending 2 Emails a Week
I’m convinced 95% of cubicle workers who work over 60 hours a week constantly can cut it down to 40-45 hours by sending 2 emails a week to their ...

Emoticon and Emoji Box for iPhone -Save Emoticons,emoji,pic and images for Sending Message! 200 FREE ...
Get Emoticon and Emoji Box for iPhone -Save Emoticons,emoji,pic and images for Sending Message! 200 FREE emoticons and emojis - on the App Store. ...

Thanks for sending an iPhone4 and Sim T-Mobile but ... - Flickr - Photo Sharing!
... that's probably not going to work. (also, I'm sticking with the iPod touch).

Texas Man Makes $10K A Month Sending Potatoes With Personalized Messages In The Mail! - YouTube
Alex Craig, 24, said he came up with the idea for Potato Parcel earlier this year while having dinner with his girlfriend. His girlfriend called ...

'It was scary': widow speaks out as Muslim cleric pleads guilty to sending offensive letters to families ...
The young widow of fallen Australian soldier Brett Till still recalls the fear that came after she opened a letter sent to their home that labelled ...

Resources last updated: 2/4/2016 2:00:39 AM