Unix Programming - multithreading and sockets in C

Author

multithreading and sockets in C

husmike@gmail.com

2006-03-18, 3:32 am

Hello all
with some simple stream sockets, tcp/ip , I am
able to do the data transfer between two hosts.
But the data is really large and I am looking at
things like multithreading (which I am fairly new to) so
that I could fasten the data transfer. The existing
code used blocking sockets.
There is only one server which sends large data in the order
of GB's (~10) to one client which requests the data.
So does multithreading help me in faster data transfer?
It would be great if anyone could send some links
or sample sources where multithreading is implemented
with sockets.
I have been looking for some stuff online but was of no help
because most of them used MFC classes.
The environment I am working is Unix.
Thank you all in advance

Mike

Brian C

2006-03-18, 3:32 am

husmike@gmail.com wrote:
> Hello all
> with some simple stream sockets, tcp/ip , I am
> able to do the data transfer between two hosts.
> But the data is really large and I am looking at
> things like multithreading (which I am fairly new to) so
> that I could fasten the data transfer. The existing
> code used blocking sockets.
> There is only one server which sends large data in the order
> of GB's (~10) to one client which requests the data.
> So does multithreading help me in faster data transfer?
> It would be great if anyone could send some links
> or sample sources where multithreading is implemented
> with sockets.
> I have been looking for some stuff online but was of no help
> because most of them used MFC classes.
> The environment I am working is Unix.
> Thank you all in advance
>
> Mike
>
Multithreading would allow your process to handle multiple connections.
By simply running multithreaded, it will not speed up your transfer. You
will obviously be doing a lot of blocking as the data is sent.

What you could do, which is similair to BitTorrent and other file
sharing protocols, is have multiple threads running, each sending a
different part of the file, which is then reconstructed on the client
side, which also must be running multiple threads (or instances), and
know how to reconstruct the file (i.e. some kind of header before
sending saying this data starts at offset 1,073,741,824 (1GB), etc.

There is really no special multithreading socket code. You pretty much
just accept() the connection, spin off a thread, passing it the socket
handle it should work on, and say what start byte position/end position,
for example:

typedef struct
{
char Filename[MAXPATH];
unsigned long StartPos;
unsigned long EndPos;
int Socket;
} ThreadArgs;

int StartSendingThread(ThreadArgs *Args)
{
pthread_t tid;
pthread_attr_t attr;

pthread_attr_init(&attr);

pthread_attr_setdetachstate(&attr,PTHREAD_CREATE_DETACHED);
pthread_create(&tid, &attr, SendingThread, Args);
}

void *SendingThread(void *ThrArgs)
{
ThreadArgs *Args;
unsigned char ReadBuffer[4096];
int BytesRead;
FILE *stream;

Args=(ThreadArgs *)ThrArgs;

if((stream=fopen(Filename,"rb"))==NULL)
{
fprintf(stderr,"Error: cannot open '%s'.\n",
Filename);
return(NULL);
}

fseek(stream,Args->StartPos,SEEK_SET);
while(ftell(stream<Args->EndPos))
{
BytesRead=fread(ReadBuffer,1,sizeof(Read
Buffer),stream);
if(WriteMsgToSocket(Args->Socket,
ReadBuffer,
BytesRead))
{
fprintf(stderr,"Error: cannot send data to socket.\n");
fseek(stream,0L,SEEK_END);
continue;
}
}

}

int WriteMsgToSocket(int Socket,unsigned char *Msg,int Bytes)
{
int BytesLeftToWrite;
int BytesWritten;
unsigned char *MsgPtr;

BytesLeftToWrite=Bytes;
MsgPtr=Msg;

while (BytesLeftToWrite>0)
{
BytesWritten=write(Socket,MsgPtr,BytesLe
ftToWrite);

if(BytesWritten<=0)
{
fprintf(stderr,"Error: write() returned %d <%s:%d>\n",
BytesWritten,
strerror(errno),
errno);
return(BytesWritten);
}

BytesLeftToWrite-=BytesWritten;
MsgPtr+=BytesWritten;
}

return(Bytes-BytesLeftToWrite);
}

Hope this helps. There may be some small errors, but I just typed this
up now.

Maxim Yegorushkin

2006-03-19, 12:02 pm

Brian C wrote:
> husmike@gmail.com wrote:

[]

[]
[vbcol=seagreen]
> What you could do, which is similair to BitTorrent and other file
> sharing protocols, is have multiple threads running, each sending a
> different part of the file, which is then reconstructed on the client
> side, which also must be running multiple threads (or instances), and
> know how to reconstruct the file (i.e. some kind of header before
> sending saying this data starts at offset 1,073,741,824 (1GB), etc.

This is unlikely to speed up data transfer. This way you share the same
data link between more TCP streams adding more overhead. BitTorrent and
other p2p clients do that in order to download files from multiple
sources rather than one.

Brian C

2006-03-19, 12:02 pm

Maxim Yegorushkin wrote:
> Brian C wrote:
>
>
>
> []
>
>
>
>
> []
>
>
>
>
> This is unlikely to speed up data transfer. This way you share the same
> data link between more TCP streams adding more overhead. BitTorrent and
> other p2p clients do that in order to download files from multiple
> sources rather than one.
>
True, I forgot about that.

phil-news-nospam@ipal.net

2006-03-19, 12:02 pm

On 17 Mar 2006 19:19:11 -0800 husmike@gmail.com wrote:

| with some simple stream sockets, tcp/ip , I am
| able to do the data transfer between two hosts.
| But the data is really large and I am looking at
| things like multithreading (which I am fairly new to) so
| that I could fasten the data transfer. The existing
| code used blocking sockets.
| There is only one server which sends large data in the order
| of GB's (~10) to one client which requests the data.
| So does multithreading help me in faster data transfer?

The client machine should be able to read data as fast as it arrives
up to the point where it runs out of CPU cycles, memory bandwidth,
buffer space, or interface speed. Going to multithreading would not
help any more than just making sure your program is doing things the
right way.

Multithreading is useful when you have an actual need to concurrently
run many contexts, and need to either have a lot of shared data, or
in the case of an extreme number of contexts, need to avoid much of
the overhead of context switching.

If your process that is receiving data is being slowed down by doing
other things between each chunk of read, then that could slow things
down. Given the large volume of data, I doubt you are stuffing it
all into memory. More likely you are writing it to a file. You may
get some speed improvement by being better at writing. If your OS
supports options to speed things up (O_DIRECT in Linux), you might
try them.

| It would be great if anyone could send some links
| or sample sources where multithreading is implemented
| with sockets.

No different than anything else. But I don't think multithreading
is, by itself, going to speed things up. But you may get some speedup
if one thread is reading the socket and filling in a ring buffer, and
the other thread is grabbing data from the ring buffer when there is
a page size minimum available, and writing to disk. Since updates to
ring buffers are generally not atomic, you'll need to get exclusive
locks when changes are made every time data is put in or taken out.

I once wrote a program that takes Apache log data coming over a pipe
and writing it to selected files based on content. One issue was to
make sure my program didn't end up blocking Apache processes on the
pipe. But I found it quite adequate to just have a process that reads
one pipe (from Apache) and writes another pipe (goes to the analyzer
part of my program) with everything queued in a few megabytes of ring
buffer. Pipes and processes and "loosen" things up a bit where they
may otherwise be "sticky" when doing too many different things at once.
I just don't see threads as any better a solution, and quite possibly
more of a headache if you don't have experience with them. If you did
have extensive experience with threads, you wouldn't be asking this.

| I have been looking for some stuff online but was of no help
| because most of them used MFC classes.
| The environment I am working is Unix.

Can you be more specific on the OS? Or is this to be portable?

If your systems and server can support SCTP in place of TCP, you may
be able to get some speedup with that. It would be a lot more coding.

--
-----------------------------------------------------------------------------
| Phil Howard KA9WGN | http://linuxhomepage.com/ http://ham.org/ |
| (first name) at ipal.net | http://phil.ipal.org/ http://ka9wgn.ham.org/ |
-----------------------------------------------------------------------------

Chris Friesen

2006-03-21, 3:17 am

husmike@gmail.com wrote:

> There is only one server which sends large data in the order
> of GB's (~10) to one client which requests the data.

Multithreading likely won't help. However, there are other things which
could, assuming that you're bottleneck is the network. How much
bandwidth do you have?

1) Is the data at all similar to the previous time you sent it? You
might be able to save time by only sending the differences.

2) Is the data at all compressible? If so, compressing it (using gzip
or bzip2) before sending might save a lot of time.

Chris