|
Home > Archive > Unix Shell > March 2006 > Need to convert DOS to UNIX with a *limited* set of UNIX commands
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Need to convert DOS to UNIX with a *limited* set of UNIX commands
|
|
| steve.chambers@gmail.com 2006-03-17, 5:54 pm |
| I'm stuck! Need to do a text replace in files to convert them from DOS
into equivalent UNIX format (i.e. get rid of all the ^M characters).
But am using a system that has a BusyBox UNIX implementation on it that
supports a very limited set of unix commands. These are:
alias break builtin cd chdir continue eval exec exit export
false fc hash help jobs let local pwd read readonly return set
setvar shift times trap true type ulimit umask unalias unset
wait [ ash basename busybox cat chmod cp cut df dmesg du echo
env false find free freeramdisk halt hostid hostname init kill
killall ln logger ls mkdir mknod mktemp more mount mv nc nslookup
ps pwd reboot rm rmdir sh sleep sync test tftp touch traceroute
true umount uname uniq uptime
Can anyone see a way to do it from the command line automatically with
these commands? I'm aware that I could do it in Vi (which is also
available but not listed) but need it to be done at the command line
automatically as part of a script rather than as a manual process. Just
in case anyone suggests it, the system doesn't have any implementation
of SED, EMACS or TR.
Any advice on this would be much appreciated!
Cheers,
Steve
| |
| Eric Moors 2006-03-17, 5:54 pm |
| steve.chambers@gmail.com wrote:
> I'm stuck! Need to do a text replace in files to convert them from DOS
> into equivalent UNIX format (i.e. get rid of all the ^M characters).
> But am using a system that has a BusyBox UNIX implementation on it that
> supports a very limited set of unix commands. These are:
>
> alias break builtin cd chdir continue eval exec exit export
> false fc hash help jobs let local pwd read readonly return set
> setvar shift times trap true type ulimit umask unalias unset
> wait [ ash basename busybox cat chmod cp cut df dmesg du echo
> env false find free freeramdisk halt hostid hostname init kill
> killall ln logger ls mkdir mknod mktemp more mount mv nc nslookup
> ps pwd reboot rm rmdir sh sleep sync test tftp touch traceroute
> true umount uname uniq uptime
>
> Can anyone see a way to do it from the command line automatically with
> these commands? I'm aware that I could do it in Vi (which is also
> available but not listed) but need it to be done at the command line
> automatically as part of a script rather than as a manual process. Just
> in case anyone suggests it, the system doesn't have any implementation
> of SED, EMACS or TR.
>
> Any advice on this would be much appreciated!
>
As you appear to have cut available, you could try this:
cut -d ^M -f1 inputFile > outputfile
Eric
| |
| steve.chambers@gmail.com 2006-03-17, 5:54 pm |
| Hmmm, good suggestion. It may *possibly* work but unfortunately I'm not
quite there with it yet. Tried the line as suggested and it gave the
message of "cut: the delimiter must be a single character". So I then
tried using \r instead of ^M and it simply removed all the ends of
lines with r in them from the r onwards. I then played about a bit and
discovered that if I do a range (e.g. -f1-5) then it includes the
delimiters in the output (in this case the 'r' character) but if I do a
single field e.g. f1, f2 etc. it doesn't. So not sure if it will be
workable as to convert into UNIX format we need to remove the \r but
keep the \n from \r\n. Aaarrgh, frustrating!
So in summary I guess there's two new questions here I don't know the
answer to:-
1. How do I specify the carriage return character? (have tried ^M, \r,
"^M", "\r", \\r, "\\r" but none of these work)
2. Would there be any way of using cut to not include the delimiters in
order to preserve the \n?
Not looking too hopeful...
Oh, one other thing, I found where the other commands that are
available are in /usr/bin folder. They are:
[ erase killall nc uniq
basename eraseall lock nftl_format unlock
cut fcp logger nftldump uptime
doc_loadbios free mkfs.jffs nslookup vi
du ftl_check mtd_debug passwd
einfo ftl_format nanddump test
elvis grep nandtest tftp
env hostid nandwrite traceroute
Probably not much help for solving this but hopefully someone might
disagree...?
| |
| Stephane Chazelas 2006-03-17, 5:54 pm |
| On 17 Mar 2006 08:03:01 -0800, steve.chambers@gmail.com wrote:
> Hmmm, good suggestion. It may *possibly* work but unfortunately I'm not
> quite there with it yet. Tried the line as suggested and it gave the
> message of "cut: the delimiter must be a single character".
[...]
The suggestion was:
cut -d^M -f1
where ^M was the CR character, not the two character ^ and M. To
type that character at the prompt, type <Ctrl-V> followed by
<Ctrl-M> (or <Return> ).
Alternatively, you should be able to do either:
CR=`printf '\r'`
or
CR=`echo '\r'`
or
CR=`echo -e '\r'`
cut -d "$CR" -f1
--
Stephane
| |
| Chris F.A. Johnson 2006-03-17, 5:54 pm |
| On 2006-03-17, steve.chambers@gmail.com wrote:
> Hmmm, good suggestion.
What is? Please quote context.
> It may *possibly* work but unfortunately I'm not
> quite there with it yet. Tried the line as suggested and it gave the
> message of "cut: the delimiter must be a single character". So I then
> tried using \r instead of ^M and it simply removed all the ends of
> lines with r in them from the r onwards. I then played about a bit and
> discovered that if I do a range (e.g. -f1-5) then it includes the
> delimiters in the output (in this case the 'r' character) but if I do a
> single field e.g. f1, f2 etc. it doesn't. So not sure if it will be
> workable as to convert into UNIX format we need to remove the \r but
> keep the \n from \r\n. Aaarrgh, frustrating!
>
> So in summary I guess there's two new questions here I don't know the
> answer to:-
>
> 1. How do I specify the carriage return character? (have tried ^M, \r,
> "^M", "\r", \\r, "\\r" but none of these work)
Press control-V then control-M
> 2. Would there be any way of using cut to not include the delimiters in
> order to preserve the \n?
cut -d '^M' -f1 FILE
--
When answering to a Usenet post through Google groups, please click
on "show options" at the top of the article, then click on the
"Reply" in the article headers. (see <http://cfaj.freeshell.org/google> ),
------ Chris F.A. Johnson, <http://cfaj.freeshell.org> ------
| |
| David Douthitt 2006-03-17, 5:54 pm |
| On 2006-03-17 08:33:54 -0600, steve.chambers@gmail.com said:
> I'm stuck! Need to do a text replace in files to convert them from DOS
> into equivalent UNIX format (i.e. get rid of all the ^M characters).
> But am using a system that has a BusyBox UNIX implementation on it that
> supports a very limited set of unix commands.
Would cat work? Like this:
cat $FILE > $FILE.new
This is predicated on the fact that cat converts line-endings to
standard UNIX line endings; I know less can do this, but it doesn't
seem to be present.
If you do a cat, and everything looks just fine, then redirect it to a
new file....
| |
| laura fairhead 2006-03-17, 5:54 pm |
| On 17 Mar 2006 08:03:01 -0800, steve.chambers@gmail.com wrote:
>Hmmm, good suggestion. It may *possibly* work but unfortunately I'm not
>quite there with it yet. Tried the line as suggested and it gave the
>message of "cut: the delimiter must be a single character". So I then
>tried using \r instead of ^M and it simply removed all the ends of
>lines with r in them from the r onwards. I then played about a bit and
>discovered that if I do a range (e.g. -f1-5) then it includes the
>delimiters in the output (in this case the 'r' character) but if I do a
>single field e.g. f1, f2 etc. it doesn't. So not sure if it will be
>workable as to convert into UNIX format we need to remove the \r but
>keep the \n from \r\n. Aaarrgh, frustrating!
>
>So in summary I guess there's two new questions here I don't know the
>answer to:-
>
>1. How do I specify the carriage return character? (have tried ^M, \r,
>"^M", "\r", \\r, "\\r" but none of these work)
Hiya,
Type the lnext key (Literal Next - usually this is CTRL+V ) then CTRL+M
then the 'cut' method Eric described should work: '^M' does not mean
CARET-M, it is a representation of CTRL+M character code 13.
(notice how M is the 13th letter in the alphabet and this is character
code 13, means you can quickly memorise these codes if you know the
numbers of the letters in the alphabet well I digress... ;-)
>2. Would there be any way of using cut to not include the delimiters in
>order to preserve the \n?
>
>Not looking too hopeful...
*smile* there is _always_ a way!
>
>Oh, one other thing, I found where the other commands that are
>available are in /usr/bin folder. They are:
>
>[ erase killall nc uniq
>basename eraseall lock nftl_format unlock
>cut fcp logger nftldump uptime
>doc_loadbios free mkfs.jffs nslookup vi
>du ftl_check mtd_debug passwd
>einfo ftl_format nanddump test
>elvis grep nandtest tftp
>env hostid nandwrite traceroute
>
>Probably not much help for solving this but hopefully someone might
>disagree...?
Well 'vi' would probably be the natural choice here, real vi should do
it with a simple :%s/.$// command but it's probably vim in which case
I cannot tell you offhand but a quick trawl through the manual pages
should reveal it.... Another way of doing it would be a quick shell
script if the 'read' command supported raw '-r' but I really really
don't think you're going to need to have to go that far here <g>
bestwishesfrom
laura
>
--
echo moc.12klat@daehriaf_arual|sed 's/\(.\)\(.\),*/\2,\1/g;h;s/,//g;/@t/q;G;D'
| |
| Icarus Sparry 2006-03-17, 5:54 pm |
| On Fri, 17 Mar 2006 06:33:54 -0800, steve.chambers wrote:
> I'm stuck! Need to do a text replace in files to convert them from DOS
> into equivalent UNIX format (i.e. get rid of all the ^M characters).
> But am using a system that has a BusyBox UNIX implementation on it that
> supports a very limited set of unix commands. These are:
>
> alias break builtin cd chdir continue eval exec exit export
> false fc hash help jobs let local pwd read readonly return set
> setvar shift times trap true type ulimit umask unalias unset
> wait [ ash basename busybox cat chmod cp cut df dmesg du echo
> env false find free freeramdisk halt hostid hostname init kill
> killall ln logger ls mkdir mknod mktemp more mount mv nc nslookup
> ps pwd reboot rm rmdir sh sleep sync test tftp touch traceroute
> true umount uname uniq uptime
>
> Can anyone see a way to do it from the command line automatically with
> these commands? I'm aware that I could do it in Vi (which is also
> available but not listed) but need it to be done at the command line
> automatically as part of a script rather than as a manual process. Just
> in case anyone suggests it, the system doesn't have any implementation
> of SED, EMACS or TR.
>
> Any advice on this would be much appreciated!
>
> Cheers,
> Steve
In addition to the suggestions you have had for using 'cut', if you have
'vi' then you almost certainly have 'ex', which can be scripted without
any problems. (You may need to use 'vi -e' or some other incantation).
You may also be able to do the task just using a loop, reading lines and
echoing them, if you set IFS to a control-M character, e.g.
#!/bin/sh
IFS="^M" # type control-v control-m to get the ^M character
while read line
do
echo "$line"
done
season with options that are available, e.g. '-r' to the read, '-E' to the
echo, to reduce the amount of interpretation that the commands do.
| |
| Jordan Abel 2006-03-17, 5:54 pm |
| On 2006-03-17, steve.chambers@gmail.com <steve.chambers@gmail.com> wrote:
> Hmmm, good suggestion. It may *possibly* work but unfortunately I'm not
> quite there with it yet. Tried the line as suggested and it gave the
> message of "cut: the delimiter must be a single character". So I then
> tried using \r instead of ^M and it simply removed all the ends of
> lines with r in them from the r onwards. I then played about a bit and
> discovered that if I do a range (e.g. -f1-5) then it includes the
> delimiters in the output (in this case the 'r' character) but if I do a
> single field e.g. f1, f2 etc. it doesn't. So not sure if it will be
> workable as to convert into UNIX format we need to remove the \r but
> keep the \n from \r\n. Aaarrgh, frustrating!
>
> So in summary I guess there's two new questions here I don't know the
> answer to:-
>
> 1. How do I specify the carriage return character? (have tried ^M, \r,
> "^M", "\r", \\r, "\\r" but none of these work)
press control-v and then enter.
> 2. Would there be any way of using cut to not include the delimiters in
> order to preserve the \n?
>
> Not looking too hopeful...
>
> Oh, one other thing, I found where the other commands that are
> available are in /usr/bin folder. They are:
>
> [ erase killall nc uniq
> basename eraseall lock nftl_format unlock
> cut fcp logger nftldump uptime
> doc_loadbios free mkfs.jffs nslookup vi
> du ftl_check mtd_debug passwd
> einfo ftl_format nanddump test
> elvis grep nandtest tftp
> env hostid nandwrite traceroute
>
> Probably not much help for solving this but hopefully someone might
> disagree...?
elvis/vi may be capable of doing the transformation interactively.
| |
| Jeremiah DeWitt Weiner 2006-03-17, 5:54 pm |
| David Douthitt <ssrat@mailbag.com> wrote:
> Would cat work? Like this:
> cat $FILE > $FILE.new
> This is predicated on the fact that cat converts line-endings to
> standard UNIX line endings
No, it doesn't. They may not appear as funny characters on your
screen, but cat should be binary-safe, so it should do no conversion at
all. Observe:
selene:/tmp$ file dosfile.txt
dosfile.txt: ASCII text, with CRLF line terminators
selene:/tmp$ cat -v dosfile.txt
Hello, this is^M
a file with DOS^M
line endings.^M
selene:/tmp$ cat dosfile.txt > dosfile.txt.new
selene:/tmp$ file dosfile.txt.new
dosfile.txt.new: ASCII text, with CRLF line terminators
selene:/tmp$ md5sum dosfile.txt*
f2df24a53b6cf7a48a60f4d1f1ba1fef dosfile.txt
f2df24a53b6cf7a48a60f4d1f1ba1fef dosfile.txt.new
--
Oh to have a lodge in some vast wilderness. Where rumors of oppression
and deceit, of unsuccessful and successful wars may never reach me
anymore.
-- William Cowper
| |
| Bill Marcum 2006-03-18, 3:31 am |
| On Fri, 17 Mar 2006 10:32:10 -0600, David Douthitt
<ssrat@mailbag.com> wrote:
> On 2006-03-17 08:33:54 -0600, steve.chambers@gmail.com said:
>
>
> Would cat work? Like this:
>
> cat $FILE > $FILE.new
>
Try it and see, but as far as I know, cat does an exact copy, other than
the options listed in 'man cat'. Also, if your busybox doesn't include
sed or tr, is there some reason you can't compile a better one?
--
The future lies ahead.
| |
| joeschmoe40@yahoo.com 2006-03-19, 12:02 pm |
|
Bill Marcum wrote:
....
> Try it and see, but as far as I know, cat does an exact copy, other than
> the options listed in 'man cat'. Also, if your busybox doesn't include
> sed or tr, is there some reason you can't compile a better one?
Well, the standard in these sorts of problems ("I have to do X, using
only tools Y & Z", i.e., the equivalent of the usual "straight-edge &
compass" geometry problems) is that if you can compile anything, then
that ruins the problem (aka, "that's cheating!")
As an aside, I often wish that when people post these "straight-edge &
compass" problems ("I can't compile anything"), that they'd tell us
*why* they can't. I.e., it is usually one of the following:
1) Because it *is* an intellectual exercise (classic "straight-edge &
compass")
2) Because management says they can't. And, by the way, if it is this,
then the answer is "Fix management". Trust me, it can be done, and it
*is* the right answer.
3) Because they themselves lack the experience/knowledge to work a
compiler. And, it is this, then the answer is "Educate thyself - it
will make you a better person".
| |
| Robert Bonomi 2006-03-21, 3:17 am |
| In article <1142703802.447431.233860@i39g2000cwa.googlegroups.com>,
<joeschmoe40@yahoo.com> wrote:
>
>Bill Marcum wrote:
>...
>
>Well, the standard in these sorts of problems ("I have to do X, using
>only tools Y & Z", i.e., the equivalent of the usual "straight-edge &
>compass" geometry problems) is that if you can compile anything, then
>that ruins the problem (aka, "that's cheating!")
>
>As an aside, I often wish that when people post these "straight-edge &
>compass" problems ("I can't compile anything"), that they'd tell us
>*why* they can't. I.e., it is usually one of the following:
>
>1) Because it *is* an intellectual exercise (classic "straight-edge &
>compass")
>
>2) Because management says they can't. And, by the way, if it is this,
>then the answer is "Fix management". Trust me, it can be done, and it
>*is* the right answer.
>
>3) Because they themselves lack the experience/knowledge to work a
>compiler. And, it is this, then the answer is "Educate thyself - it
>will make you a better person".
>
you forgot:
4) it is a "run-time only" environment. e.g. old SCO XENIX _without_
the 'development' toolset (an extra-cost option).
It looks like the OP's situation falls into category 4. Doesn't have
_any_ of what are generally considered UNIX development tools. Looks like
a 'bare minimums' set of utilities to boot up and 'error fix' the system
into multi-user mode, and to edit system (and the dedicated application that
the box -- and pre-loaded kernel -- was designed to support) configuration
files.
There _is_ something to be said for this kind of approach -- there are _way_
fewer things that a 'less than knowledgeable user' can screw up, when they
don't have access to tools to MIS-use. <wry grin>
It's a d*mn nuisance when you need to do something the vendor didn't provide
tools for, BUT, at the same time, the fact that there's no tool means that
you're probably trying to do things the "wrong way", or you're into something
that you "shouldn't be" doing. And the way to get the job done *while*
ensuring that the system (and that 'critical' application running on it)
remains _stable_ is to *use* your 'support contract', and have the *vendor*
do it. Who _does_ have access to the other tools, and does "know what they
are doing", and (hopefully! <wry grin> ) will not screw the production system
up. Even better, if they _do_ screw up, you've got "somebody else" to blame,
and a target for the lawsuit to recover the losses that resulted from that
screw up. If the 'source' of the problem is in-house, the only 'recourse'
is to "fire somebody". Which does _not_ 'make the company whole'.
| |
| steve.chambers@gmail.com 2006-03-21, 3:17 am |
| Fantastic!
Thanks a lot Stephane, was beginning to lose hope when the CTRL-V
didn't work and nor did the printf or first echo commands but the final
variable definition with the -e option sorted it, I'm well chuffed with
that.
Also a big thanks to everyone else for their suggestions, didn't really
expect to receive any replies, must have underestimated the helpfulness
of the online UNIX community!
|
|
|
|
|