Hello, I'm writing  a program for sending images over a socket.

The size of the image is 640x480 pixels in pgm format.

The problem is that when the client sends the image to the server (307200
bytes) the server only receives 65533.

The thing is, is there any limits on the information sent from one point
(send) to another (recv)?.
In such case, how is the way I have to send one image of 640x480 pixels?
Perhaps in two times?

Thank you for your attention,
Joaquin.

[ Post a follow-up to this message ]

Joaquin López wrote:
quote:

> Hello, I'm writing  a program for sending images over a socket.
>
> The size of the image is 640x480 pixels in pgm format.
>
> The problem is that when the client sends the image to the server (307200
> bytes) the server only receives 65533.
>
> The thing is, is there any limits on the information sent from one point
> (send) to another (recv)?.
> In such case, how is the way I have to send one image of 640x480 pixels?
> Perhaps in two times?
>
> Thank you for your attention,
> Joaquin.
>
>


Let me guess: you wrote the image to the socket with a single write
statement?  And now you expect to read the image with a single read??

That's now how it is done.  The transport (the stuff behind the socket)
cannot handle messages of that size, and it breaks up the message into
pieces which are then sent to the receiving socket.  When you read, you
will only get part of the message, and so you must repeatedly read the
socket until you get all of the expected data.  How, you ask, do you
know when you have all of the data??  You know because you sent the
length of the message before you sent the message.

--

Fletcher Glenn

[ Post a follow-up to this message ]

On Mon, 8 Dec 2003 17:14:30 +0100, "Joaquin López" <lsanchez@lsi.uji.es> wro
te:
quote:

>Hello, I'm writing  a program for sending images over a socket.
>
>The size of the image is 640x480 pixels in pgm format.
>
>The problem is that when the client sends the image to the server (307200
>bytes) the server only receives 65533.
>
>The thing is, is there any limits on the information sent from one point
>(send) to another (recv)?.
>
>In such case, how is the way I have to send one image of 640x480 pixels?
>Perhaps in two times?


It is likely that you don't understand TCP communications and/or stream sock
ets.

A recv() call only returns the data received up to that point. There is no
synchronization between the data sent and the data received. That is to say 
that
although you may /send/ massive amounts of data in one call, the correspondi
ng
recv() call does not have to (and usually won't) receive the data in one rec
v()
call. This is the nature of TCP/IP, and is not a flaw in the sockets
implementation, or anything else.

It appears that your /first/ recv() gets 64k of the data sent. You need to
continue issueing recv() calls (and appending the results together) until yo
u
receive all the data you sent. How you determine that you have received /all
/
the data depends on your sending program:
- It may close it's sending socket when it has sent all the data; the receiv
er
will detect this as an end-of-file on the receiving socket, or
- It may send the length of the data as a data item prior to the data itself
;
the receiver must fetch this length first, then recv() until the length has
been exhausted, or
- It may send a sentinal in the data stream to mark the end of the data; the
receiver must be sensitive to this sentinal and trim it's recv()d data
according to the placement of the sentinal, or
- The sender and receiver always know exactly how much data is transferred a
s
the sender always sends a fixed amount of data, and the receiver knows what
that fixed amount is; both sender and receiver must agree on what this amoun
t
is prior to the transmission of the data.

In all cases, both sender application and receiver application must agree up
on
the strategy to be used to detect end of data, and both applications must be
coded to enact that strategy.


--
Lew Pitcher
IT Consultant, Enterprise Technology Solutions
Toronto Dominion Bank Financial Group

(Opinions expressed are my own, not my employers')

[ Post a follow-up to this message ]

In article <3fd4a376@news.vodafone.es>,
"Joaquin López" <lsanchez@lsi.uji.es> wrote:
quote:

> Hello, I'm writing  a program for sending images over a socket.
>
> The size of the image is 640x480 pixels in pgm format.
>
> The problem is that when the client sends the image to the server (307200
> bytes) the server only receives 65533.
>
> The thing is, is there any limits on the information sent from one point
> (send) to another (recv)?.
> In such case, how is the way I have to send one image of 640x480 pixels?
> Perhaps in two times?


There's no limit to how much you can send over a TCP socket.  Large web
pages and huge emails are sent routinely this way.

It sounds like you're not calling recv() enough.  You have to call it in
a loop until you get everything.  TCP is a byte stream transport, it
doesn't keep track of message boundaries.  So there's no reason to
expect that everything sent in a single send() call will be returned by
a single recv() call.

--
Barry Margolin, barmar@alum.mit.edu
Woburn, MA

[ Post a follow-up to this message ]

"Joaquin López" <lsanchez@lsi.uji.es> wrote in message
news:3fd4a376@news.vodafone.es...
quote:

> Hello, I'm writing  a program for sending images over a socket.
>
> The size of the image is 640x480 pixels in pgm format.
>
> The problem is that when the client sends the image to the server (307200
> bytes) the server only receives 65533.


Put the read in a loop.  If you do a tcpdump

tcpdump ether src port <your port>

Look for the mss value "maximum segment size" in the 3 way hand, which
should be the number of bytes for each call to read.

/** server side **/
process_stuff(int sockfd)
{
ssize_t n;
char line[MAXLINE];

sockfd_to_family(sockfd);
while(1) {
if ( ( n = read( sockfd, line, MAXLINE)) == 0 )
{
fprintf(stderr,"client had nothing else to say \n");
return;
}
line[n]='\0';
fprintf(stderr,"client said :n=%d %s\n",n,line);
} /* end while */
}

When the client is done sending, it can exit and you can look for the
WNOHANG
/* server side */
sig_chld(int signo)
{
pid_t pid;
int stat;

while( (pid = waitpid ( -1, &stat, WNOHANG)) > 0 )
fprintf(stderr,"child %d terminated \n",pid);  /* not recommended to
have a print here */
return;

}

/******* start: Complete toy server for text******************/
#include <stdlib.h>
#include <arpa/inet.h>
#include <assert.h>
#include <errno.h>
#include <netinet/in.h>
#include <signal.h>
#include <stdio.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <sys/wait.h>
#include <unistd.h>

#define SA      struct sockaddr  /* keeps the lines shorter */
#define MAXLINE 4096
/* Following could be derived from SOMAXCONN in <sys/socket.h>, but many
kernels still #define it as 5, while actually supporting many more */
#define LISTENQ         1024    /* 2nd argument to listen() */
#define MAXSOCKADDR  128        /* max socket address structure size */

/* normally these would go in a header */
void sig_chld(int);
void process_stuff(int);
int sockfd_to_family(int );

/*
The follwing function prevents zombie or defunct processes:
which occurs with server2way when the client is killed:


$ps -U chirico -auf
chirico  30923  0.0  0.0  1352  252 pts/0    S    11:02   0:00  \_
./server2way
chirico  30924  0.0  0.0     0    0 pts/0    Z    11:02   0:00      \_
[server2way <defunct>]

So, the function below...prevents the problem.

*/

void
sig_chld(int signo)
{
pid_t pid;
int stat;

while( (pid = waitpid ( -1, &stat, WNOHANG)) > 0 )
fprintf(stderr,"child %d terminated \n",pid);  /* not recommended to
have a print here */
return;

}


void
process_stuff(int sockfd)
{
ssize_t n;
char line[MAXLINE];


sockfd_to_family(sockfd);
while(1) {
if ( ( n = read( sockfd, line, MAXLINE)) == 0 )
{
fprintf(stderr,"return in process_stuff ... read == 0\n");
return;

}
line[n]='\0';
fprintf(stderr,"client:n=%d %s\n",n,line);
write(sockfd, line, n);
}
}


int
sockfd_to_family(int sockfd)
{

struct sockaddr_in sa;
socklen_t len;

len = sizeof(sa);
if (getpeername(sockfd, (struct sockaddr*) &sa, &len) < 0 )
return -1;

fprintf(stderr,"client: %s: %d\n",inet_ntoa(sa.sin_addr),(int)
ntohs(sa.sin_port));

return(sa.sin_port);
}

/*

These are the major 4 steps:
socket
bind
listen
-- signal handle (optional but good if client it killed)
accept

*/


int
main(int argc, char **argv)
{
int listenfd, connfd;
pid_t childpid;
socklen_t clilen;
struct sockaddr_in cliaddr, servaddr;

listenfd = socket(AF_INET, SOCK_STREAM, 0);

bzero(&servaddr, sizeof(servaddr));
servaddr.sin_family = AF_INET;
servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
servaddr.sin_port =  htons(10000);  /* change this to your favorate port
*/

bind(listenfd, (SA *) &servaddr, sizeof(servaddr));
listen(listenfd, LISTENQ);

signal(SIGCHLD, sig_chld);

while(1)
{
clilen = sizeof(cliaddr);
connfd = accept(listenfd, (SA *) &cliaddr, &clilen);

if ( (childpid = fork()) == 0 ) {
close(listenfd);
process_stuff(connfd);
exit(0);
}
close(connfd);
}

}

/******* end: Complete toy server ******************/

Sorry about all the code; but, it's hard to understand with just pieces.

Excellent reference: "Unix Network Programming.." W. Richard Stevens.

Regards,

Mike Chirico

[ Post a follow-up to this message ]

however, since TCP is a STREAM of bytes, it is advised you use an NBO=20
long, htol() of the number of bytes to follow so your receiver knows how =

many bytes to expect.

Barry Margolin wrote:
quote:

> In article <3fd4a376@news.vodafone.es>,
>  "Joaquin L=F3pez" <lsanchez@lsi.uji.es> wrote:
>=20
>=20 
00[QUOTE] 
t[QUOTE] 
?[QUOTE] 
>=20
>=20
> There's no limit to how much you can send over a TCP socket.  Large web=

=20
quote:

> pages and huge emails are sent routinely this way.
>=20
> It sounds like you're not calling recv() enough.  You have to call it i=

n=20
quote:

> a loop until you get everything.  TCP is a byte stream transport, it=20
> doesn't keep track of message boundaries.  So there's no reason to=20
> expect that everything sent in a single send() call will be returned by=

=20
quote:

> a single recv() call.
>=20

[ Post a follow-up to this message ]

On Mon, 08 Dec 2003 17:48:46 GMT, Joseph Dionne <jdionne@sdcnov1.sdcweb.net>
wrote:
quote:

>however, since TCP is a STREAM of bytes, it is advised


by whom?
quote:

> you use an NBO=20
>long, htol() of the number of bytes to follow so your receiver knows how =
>
>many bytes to expect.


Or any one of the alternate strategies, like
- having the source send a sentinal value at the end of the data, or
- having the source close it's outbound data stream after it has written
the last byte of data, or
- having the source and receiver programs agree that the transferred data wi
ll
always be a specific, fixed number of octets. The source sends exactly that
number of octets, and the receiver reads exactly that number of octets
preprogrammed.



[snip]

--
Lew Pitcher
IT Consultant, Enterprise Technology Solutions
Toronto Dominion Bank Financial Group

(Opinions expressed are my own, not my employers')

[ Post a follow-up to this message ]

I stand corrected, Lew, any of the alternatives you have suggested will
work just fine, except perhaps disconnect which might be delivered OOB,
causing a less than full read.

However, a "common" practice for binary data transmission is the use a
length bytes, two or four bytes (I recommend NBO -- network byte order)
to eliminate the need for concern over Endian differences between client
and server.

HTML, one of the most popular STREAMs over sockets is token, and "line",
delimited, but a "sentinel" in a binary stream would require one to use
transparency to allow the "end sentinel" appearing in the image being sent.

Just a thought.  I find lengths bytes serve two purposes; 1) the
receiver knows how much data to perhaps malloc() a buffer, 2) restores
message structure to the TCP STREAM if one plans to use the connection
multiple times.

Just a thought.

Lew Pitcher wrote:
quote:

> On Mon, 08 Dec 2003 17:48:46 GMT, Joseph Dionne <jdionne@sdcnov1.sdcweb.ne
t>
> wrote:
>
> 
>
>
> by whom?
>
> 
>
>
> Or any one of the alternate strategies, like
> - having the source send a sentinal value at the end of the data, or
> - having the source close it's outbound data stream after it has written
>   the last byte of data, or
> - having the source and receiver programs agree that the transferred data 
will
>   always be a specific, fixed number of octets. The source sends exactly t
hat
>   number of octets, and the receiver reads exactly that number of octets
>   preprogrammed.
>
>
>
> [snip]
>

[ Post a follow-up to this message ]

In article <PK3Bb.31593$b01.723209@twister.tampabay.rr.com>,
Joseph Dionne <jdionne@sdcnov1.sdcweb.net> wrote:
quote:

> I stand corrected, Lew, any of the alternatives you have suggested will
> work just fine, except perhaps disconnect which might be delivered OOB,
> causing a less than full read.


It shouldn't be delivered OOB.  Closing the connection results in an EOF
being read.

--
Barry Margolin, barmar@alum.mit.edu
Woburn, MA

[ Post a follow-up to this message ]

Lew Pitcher wrote:
quote:

> On Mon, 8 Dec 2003 17:14:30 +0100, "Joaquin López" <lsanchez@lsi.uji.es>
> wrote:
> 
>
> It is likely that you don't understand TCP communications and/or stream
> sockets.
>
> A recv() call only returns the data received up to that point. There is no
> synchronization between the data sent and the data received. That is to
> say that although you may /send/ massive amounts of data in one call, the
> corresponding recv() call does not have to (and usually won't) receive the
> data in one recv() call. This is the nature of TCP/IP, and is not a flaw
> in the sockets implementation, or anything else.
>
> It appears that your /first/ recv() gets 64k of the data sent. You need to
> continue issueing recv() calls (and appending the results together) until
> you receive all the data you sent. How you determine that you have
> received /all/ the data depends on your sending program:
> - It may close it's sending socket when it has sent all the data; the
> receiver
>   will detect this as an end-of-file on the receiving socket, or
> - It may send the length of the data as a data item prior to the data
> itself;
>   the receiver must fetch this length first, then recv() until the length
>   has been exhausted, or
> - It may send a sentinal in the data stream to mark the end of the data;
> the
>   receiver must be sensitive to this sentinal and trim it's recv()d data
>   according to the placement of the sentinal, or
> - The sender and receiver always know exactly how much data is transferred
> as
>   the sender always sends a fixed amount of data, and the receiver knows
>   what that fixed amount is; both sender and receiver must agree on what
>   this amount is prior to the transmission of the data.
>
> In all cases, both sender application and receiver application must agree
> upon the strategy to be used to detect end of data, and both applications
> must be coded to enact that strategy.
>
>


Thank you to all of us for helping me. The problem was exactly this, I
expected recv() will get all the data in only one call. It was my fault.

[ Post a follow-up to this message ]