Unix Programming - Write to executable which has done execve

Author

Write to executable which has done execve

Bronson

2007-08-12, 1:23 am

Hi there,

i am trying to make a program, which will exec another app and this
app will write to the main's program executable. I make execve
(precisely execlp) and try to open old file for writing in new
process. But I returns me ETXTBSY. Why? For example access says, that
i can write to file, but fopen fails with ETXTBSY...
What can i do with that...

Best,

Przemek

David Schwartz

2007-08-12, 1:23 am

On Aug 11, 5:30 pm, Bronson <psh...@gmail.com> wrote:

> i am trying to make a program, which will exec another app and this
> app will write to the main's program executable. I make execve
> (precisely execlp) and try to open old file for writing in new
> process. But I returns me ETXTBSY. Why? For example access says, that
> i can write to file, but fopen fails with ETXTBSY...
> What can i do with that...

You can't do it that way. You cannot modify a program's running image
by modifying the executable file. If you could, self-modifying code
would not work properly.

DS

Pascal Bourguignon

2007-08-12, 1:23 am

David Schwartz <davids@webmaster.com> writes:

> On Aug 11, 5:30 pm, Bronson <psh...@gmail.com> wrote:
>
>
> You can't do it that way. You cannot modify a program's running image
> by modifying the executable file. If you could, self-modifying code
> would not work properly.

That's not exactly the point. The problem is that when you exec a
file, it is not loaded into RAM, at least not entirely. It is memory
mapped, and the file is loaded only on demand, when you call the
functions that are not already loaded. Also, swapping out the the
program text can be avoided since we already have a copy in the
program file.

So if modifying the program file was allowed, it would break randomly
the running processes using this program.

What should be done, is to create a new executable, unlink the old one
and rename the new to the old name, so only new processes get the new
program.

Note that even for normal files you should usually do something like
that, to avoid being left with inconsistent data in the files when the
program is killed for any reason in the middle of file writting.

--
__Pascal Bourguignon__ http://www.informatimago.com/

NOTE: The most fundamental particles in this product are held
together by a "gluing" force about which little is currently known
and whose adhesive power can therefore not be permanently
guaranteed.

Bronson

2007-08-12, 1:18 pm

On 12 Sie, 03:37, Pascal Bourguignon <p...@informatimago.com> wrote:
> David Schwartz <dav...@webmaster.com> writes:
>
>
>
> That's not exactly the point. The problem is that when you exec a
> file, it is not loaded into RAM, at least not entirely. It is memory
> mapped, and the file is loaded only on demand, when you call the
> functions that are not already loaded. Also, swapping out the the
> program text can be avoided since we already have a copy in the
> program file.
>
> So if modifying the program file was allowed, it would break randomly
> the running processes using this program.
>
> What should be done, is to create a new executable, unlink the old one
> and rename the new to the old name, so only new processes get the new
> program.
>
> Note that even for normal files you should usually do something like
> that, to avoid being left with inconsistent data in the files when the
> program is killed for any reason in the middle of file writting.
>
> --
> __Pascal Bourguignon__ http://www.informatimago.com/
>
> NOTE: The most fundamental particles in this product are held
> together by a "gluing" force about which little is currently known
> and whose adhesive power can therefore not be permanently
> guaranteed.

Could you explain it more precisely... So, i have two apps : main app
and update app (i know that main app could download new version, but
it has to be done on Windows with another app, so i want to do it the
same way in Linux, because i don't want to replace the code...). Main
app execve to udpate app, update download new app to temp file. After
that it renames it to main app and delete the temp file. But how do i
unlink the old app?? I mean, i suposse that rename will fail for the
same reason, why writing to executable fails now. Correct me if i am
wrong.

Thanks,

Przemek

Pascal Bourguignon

2007-08-12, 1:18 pm

Bronson <pshemu@gmail.com> writes:

> On 12 Sie, 03:37, Pascal Bourguignon <p...@informatimago.com> wrote:
>
> Could you explain it more precisely... So, i have two apps : main app
> and update app (i know that main app could download new version, but
> it has to be done on Windows with another app, so i want to do it the
> same way in Linux, because i don't want to replace the code...). Main
> app execve to udpate app, update download new app to temp file. After
> that it renames it to main app and delete the temp file. But how do i
> unlink the old app?? I mean, i suposse that rename will fail for the
> same reason, why writing to executable fails now. Correct me if i am
> wrong.

Well, you can rely on rename(2) to unlink the old app.

Both unlink(2) and rename(2) will unlink the old app without any
problem, even if it has other links. (Open file descriptors are links
too).

--
__Pascal Bourguignon__ http://www.informatimago.com/

NOTE: The most fundamental particles in this product are held
together by a "gluing" force about which little is currently known
and whose adhesive power can therefore not be permanently
guaranteed.

Logan Shaw

2007-08-12, 1:18 pm

Bronson wrote:
> Could you explain it more precisely... So, i have two apps : main app
> and update app (i know that main app could download new version, but
> it has to be done on Windows with another app, so i want to do it the
> same way in Linux, because i don't want to replace the code...). Main
> app execve to udpate app, update download new app to temp file. After
> that it renames it to main app

That's all that's necessary right there.

When changing filenames, you are changing entries in directories.
Directory entries just map names to i-node numbers. Your main
app will have i-node number (say) 1234, and your new updated one
will have i-node number 5678. If your two files are called
"my-app" and "my-app.new", then when you rename "my-app.new"
to "my-app", you are changing the mapping from this:

my-app -> 1234
my-app.new -> 5678

to this:

my-app -> 5678

If the file with i-node number 1234 still is in use in any way,
it will continue to exist. That means it will continue to exist
if there are other hard links to it, or if it is still open. But
the next time you ask the system to open() the file "my-app", it
will get the new file, the one with i-node number 5678.

> and delete the temp file.

You don't delete the temp file. You rename the temp file so that
"steals" the name that the original file file had. The temp file
itself is not changed in any way. On directory entries that
refer to the temp file are changed.

> But how do i
> unlink the old app??

The rename() call will remove one of the links to the old app by
modifying the directory which contains the name you specified. If
that was the only remaining link (which is the normal case) and if
the file is closed (which is not true in your case), the system
will delete the old file.

In your case, since the file is open, the system will wait until
nobody has the file open and delete it then. Think of it as a
simple form of garbage collection.

> I mean, i suposse that rename will fail for the
> same reason, why writing to executable fails now.

The rename() will succeed because it isn't doing anything at all
to the executable file. It is only modifying a directory that
points to the executable file.

- Logan

Bronson

2007-08-12, 7:17 pm

On 12 Sie, 18:20, Logan Shaw <lshaw-use...@austin.rr.com> wrote:
> Bronson wrote:
>
> That's all that's necessary right there.
>
> When changing filenames, you are changing entries in directories.
> Directory entries just map names to i-node numbers. Your main
> app will have i-node number (say) 1234, and your new updated one
> will have i-node number 5678. If your two files are called
> "my-app" and "my-app.new", then when you rename "my-app.new"
> to "my-app", you are changing the mapping from this:
>
> my-app -> 1234
> my-app.new -> 5678
>
> to this:
>
> my-app -> 5678
>
> If the file with i-node number 1234 still is in use in any way,
> it will continue to exist. That means it will continue to exist
> if there are other hard links to it, or if it is still open. But
> the next time you ask the system to open() the file "my-app", it
> will get the new file, the one with i-node number 5678.
>
>
> You don't delete the temp file. You rename the temp file so that
> "steals" the name that the original file file had. The temp file
> itself is not changed in any way. On directory entries that
> refer to the temp file are changed.
>
>
> The rename() call will remove one of the links to the old app by
> modifying the directory which contains the name you specified. If
> that was the only remaining link (which is the normal case) and if
> the file is closed (which is not true in your case), the system
> will delete the old file.
>
> In your case, since the file is open, the system will wait until
> nobody has the file open and delete it then. Think of it as a
> simple form of garbage collection.
>
>
> The rename() will succeed because it isn't doing anything at all
> to the executable file. It is only modifying a directory that
> points to the executable file.
>
> - Logan

In initialization phase of my app i try to set process priority to 0.
Please tell my why this crashes after autoupdate. I didn't look at
mkstemp so far. I just created temp with fopen and then chmod it to
execute. And after i make execve on newly downloaded file, setpriority
fails with errno = 13, Permission denied. Why is that?

Thanks for all your help,

Przemek

David Schwartz

2007-08-13, 7:19 pm

On Aug 12, 5:07 pm, Bronson <psh...@gmail.com> wrote:

> In initialization phase of my app i try to set process priority to 0.
> Please tell my why this crashes after autoupdate.

What do you mean by "crashes"?

> I didn't look at
> mkstemp so far. I just created temp with fopen and then chmod it to
> execute. And after i make execve on newly downloaded file, setpriority
> fails with errno = 13, Permission denied. Why is that?

Probably because your process doesn't have the appropriate permissions
to change its priority.

DS

Bronson

2007-08-13, 7:19 pm

On 14 Sie, 00:27, David Schwartz <dav...@webmaster.com> wrote:
> On Aug 12, 5:07 pm, Bronson <psh...@gmail.com> wrote:
>
>
> What do you mean by "crashes"?
>
>
> Probably because your process doesn't have the appropriate permissions
> to change its priority.
>
> DS

Now i've got it.... Sorry for all that, it was my stupid mistake...
I did execve from another thread, not the main thread...
That is why shell was quiting and probably setpriority crashing... ;-)

Thanks for all your help.

Przemek

Frank Cusack

2007-08-14, 1:20 am

On Mon, 13 Aug 2007 16:18:03 -0700 Bronson <pshemu@gmail.com> wrote:
> Now i've got it.... Sorry for all that, it was my stupid mistake...
> I did execve from another thread, not the main thread...
> That is why shell was quiting and probably setpriority crashing... ;-)

I don't see how that matters. All threads are, by definition, equivalent.
There isn't really such a thing as a "main thread".

-frank

Bronson

2007-08-15, 1:22 pm

On 14 Sie, 02:25, Frank Cusack <fcus...@fcusack.com> wrote:
> On Mon, 13 Aug 2007 16:18:03 -0700 Bronson <psh...@gmail.com> wrote:
>
>
> I don't see how that matters. All threads are, by definition, equivalent.
> There isn't really such a thing as a "main thread".
>
> -frank

I do know exactly, but "main" i call the one, which is executed when
the program executes (for example when you do not create any thread,
the main is running). I don't know - maybe it is called process
then...
I was just curious, why shell quits when i make execv. Now i know - i
call it not from the thread, which shell is executing, and that is why
it quits (in man of execve it is written, that all other threads are
closed exept one, which executes execve). I think you now get my
point.

Thanks,

Przemek

Frank Cusack

2007-08-15, 1:22 pm

On Wed, 15 Aug 2007 06:31:39 -0700 Bronson <pshemu@gmail.com> wrote:
> On 14 Sie, 02:25, Frank Cusack <fcus...@fcusack.com> wrote:
>
> I do know exactly, but "main" i call the one, which is executed when
> the program executes (for example when you do not create any thread,
> the main is running). I don't know - maybe it is called process
> then...
> I was just curious, why shell quits when i make execv. Now i know - i
> call it not from the thread, which shell is executing, and that is why
> it quits (in man of execve it is written, that all other threads are
> closed exept one, which executes execve). I think you now get my
> point.

I think I get it. Makes sense.
-frank

Scott Lurndal

2007-08-15, 7:21 pm

Frank Cusack <fcusack@fcusack.com> writes:
>On Wed, 15 Aug 2007 06:31:39 -0700 Bronson <pshemu@gmail.com> wrote:
>
>I think I get it. Makes sense.
>-frank

It does? How? There are only a few ways that a process
executed by the shell can cause the shell itself to exit:

1) The process executes something like:
kill(getppid(), SIGTERM); or
kill(-getpgid(), SIGTERM);
2) The process exits with a non-zero return value from main and
the shell is conditioned to exit on error (e.g. set -e)
3) You're tickling a bug in the shell (highly unlikely).
4) The last reference to the controlling terminal for the
process group of which the shell is a member is closed
and hupcl is set in the stty settings for the controlling tty
(SIGHUP will be sent to the process group). Since the shell
should be holding references to /dev/tty via stdin/out/err,
this shouldn't happen, but you can try a nohup command when
executing your program to see if this is the cause.

It matters not which thread within a process issues the execve. Only
shell built-in[*] commands execute in the context of a shell 'thread';
an executable will always execute in the context of a freshly forked
process which only has a parent-child relationship with the shell.

scott

[*] ksh93 & perhaps others provide mechanisms using shared objects
to allow arbitrary built-in commands, but such commands should
neither fork(2) nor exec(2).

Frank Cusack

2007-08-15, 7:21 pm

On Wed, 15 Aug 2007 18:41:06 GMT scott@slp53.sl.home (Scott Lurndal) wrote:
> Frank Cusack <fcusack@fcusack.com> writes:
>
> It does? How? There are only a few ways that a process
> executed by the shell can cause the shell itself to exit:

Oh that was not my understanding. I thought he was doing something like:

main thread
main()
pthread_create() new thread
spawn shell execve()

because otherwise the thread aspect of the whole thing is irrelevant.

-frank

Giorgos Keramidas

2007-08-15, 7:21 pm

On Wed, 15 Aug 2007 06:31:39 -0700, Bronson <pshemu@gmail.com> wrote:
>On 14 Sie, 02:25, Frank Cusack <fcus...@fcusack.com> wrote:
>
> I do know exactly, but "main" i call the one, which is executed
> when the program executes (for example when you do not create
> any thread, the main is running).

By the time that main() is entered, the program may already have
thousands of threads. It really depends on what the threading
implementation needs to do.

> I don't know - maybe it is called process then... I was just
> curious, why shell quits when i make execv. Now i know - i call
> it not from the thread, which shell is executing, and that is
> why it quits (in man of execve it is written, that all other
> threads are closed exept one, which executes execve). I think
> you now get my point.

I haven't really watched the entire thread from its beginning,
but what is exactly the problem? It sounds like you are calling
execv() from a threaded program. AFAIK, if you do this then the
entire *process* is replaced, instead of just one thread.

- Giorgos

Bronson

2007-08-16, 7:24 pm

On 15 Sie, 21:15, Giorgos Keramidas <keram...@ceid.upatras.gr> wrote:
> On Wed, 15 Aug 2007 06:31:39 -0700, Bronson <psh...@gmail.com> wrote:
>
>
>
> By the time that main() is entered, the program may already have
> thousands of threads. It really depends on what the threading
> implementation needs to do.
>
>
> I haven't really watched the entire thread from its beginning,
> but what is exactly the problem? It sounds like you are calling
> execv() from a threaded program. AFAIK, if you do this then the
> entire *process* is replaced, instead of just one thread.
>
> - Giorgos

man execve:

* All threads other than calling thread are destroyed during an
execve. Mutexes, condition variables and other pthreads objects are
not preserved.

yes, i think you're right - it's all about processes, but when i was
doing execve from other thread, shell was quiting - and i can say
nothing more....

Regards,

Przemek