Help!

can you help me understand pipe and xargs?

 
Post new topic   General Reply to Topic (not reply to a specific post)    Forums Home -> Genreal Discussions RSS
Next:  [PATCH -v2 0/7] PCI: pcie hotplug related patch  
Author Message
www
External


Since: Jan 27, 2012
Posts: 10



PostPosted: Fri Jan 27, 2012 6:25 pm    Post subject: can you help me understand pipe and xargs?
Archived from groups: comp>os>linux>misc (more info?)

Hi,

I understand pipe using "|": the output of one command becomes the
input of the second command, for example:

ls | grep abc

will list file names or directory names which contain "abc". I guess
the effect is same as: ls *abc*

The question is why we need xargs in piping. For example, the other
day I ran into this problem: I have a file with name files.txt

cat files.txt:
a.txt
b.txt
...

I added two dots in purpose as the last line of files.txt. I want to
copy all file names listed inside files.txt to one level up. My first
try and it failed:

$cat files.txt | cp
cp: missing file operand
Try `cp --help' for more information.

My second try and it worked:
$cat files.txt | xargs cp

I don't understand what the difference between the two commands. Is
that xargs put the three lines into one line so cp command can work?
So the function of "xargs" is joining all the lines it receives into
one line?

Thank you very much.
Back to top
www
External


Since: Jan 27, 2012
Posts: 10



PostPosted: Fri Jan 27, 2012 6:52 pm    Post subject: Re: can you help me understand pipe and xargs? [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

Just one more example to show my question:
$cat a.txt
goo d
bad g

A.
$cat a.txt | tac //works great, prints out the lines inside a.txt
backwards(last line becomes the first line)
bad g
goo d

B.
cat a.txt | echo //does not work ???

C.
cat a.txt | xargs echo //works, print out the content to screen

D.
$cat a.txt | xargs tac //does not work ???
tac: cannot open `goo' for reading: No such file or directory
tac: cannot open `d' for reading: No such file or directory
tac: cannot open `bad' for reading: No such file or directory
tac: cannot open `g' for reading: No such file or directory
Back to top
framefritti
External


Since: Jan 27, 2012
Posts: 3



PostPosted: Fri Jan 27, 2012 7:30 pm    Post subject: Re: can you help me understand pipe and xargs? [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

Il giorno venerd́ 27 gennaio 2012 22:25:32 UTC+1, www ha scritto:

>
> $cat files.txt | cp
> cp: missing file operand
> Try `cp --help' for more information.
>
> My second try and it worked:
> $cat files.txt | xargs cp
>
> I don't understand what the difference between the two commands.

The pipe takes the standard output of the first command (that is, what it would be printed on the screen) and copies it to the standard input (usually connected to the keyboard) of the second command, roughly, as it was typed at the keyboard.

When you want to copy a file 'from.txt' to 'to.txt' you write

cp from.txt to.txt

and you do not use 'cp' by itself and then write the filenames interactively with the keyboard, but this is what you get when you use the pipe.

If you want the output of 'cat files.txt' to be used as _parameters_ (not standard input) of cp, you can use the _backticks_

cp `cat files.txt`

In this case the shell runs the command 'cat files.txt', intercepts the standard output and uses the result as parameters to 'cp'.

So, why xargs and not backticks? To be honest, I never used xargs, according to what I found on the Wikipedia http://en.wikipedia.org/wiki/Xargs

xargs is useful when the parameter list would get too long to be acceptable by the shell, xargs split the parameter list in many smaller lists and iterates over the smaller lists (so I am afraid that with 'cp' it would not give the expected result)

>
> Thank you very much.
Back to top
Richard Kettlewell
External


Since: Feb 19, 2005
Posts: 217



PostPosted: Fri Jan 27, 2012 11:10 pm    Post subject: Re: can you help me understand pipe and xargs? [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

www writes:
> I don't understand what the difference between the two commands. Is
> that xargs put the three lines into one line so cp command can work?
> So the function of "xargs" is joining all the lines it receives into
> one line?

Briefly, the distinction you are missing is between input and arguments.
They are (in general) separate; cp never reads its input, only its
arguments.

The point of xargs is to convert input into arguments.

--
http://www.greenend.org.uk/rjk/
Back to top
Rui Maciel
External


Since: May 22, 2005
Posts: 102



PostPosted: Sat Jan 28, 2012 12:10 am    Post subject: Re: can you help me understand pipe and xargs? [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

www wrote:

> Just one more example to show my question:
> $cat a.txt
> goo d
> bad g
>
> A.
> $cat a.txt | tac //works great, prints out the lines inside a.txt
> backwards(last line becomes the first line)
> bad g
> goo d
>
> B.
> cat a.txt | echo //does not work ???
>
> C.
> cat a.txt | xargs echo //works, print out the content to screen
>
> D.
> $cat a.txt | xargs tac //does not work ???
> tac: cannot open `goo' for reading: No such file or directory
> tac: cannot open `d' for reading: No such file or directory
> tac: cannot open `bad' for reading: No such file or directory
> tac: cannot open `g' for reading: No such file or directory


rui@Kubuntu:tmp$ cat a.txt
goo d
bad g
rui@Kubuntu:tmp$ cat a.txt | xargs echo
goo d bad g
rui@Kubuntu:tmp$ echo `cat a.txt`
goo d bad g


Rui Maciel
Back to top
Robert Heller
External


Since: Sep 13, 2006
Posts: 311



PostPosted: Sat Jan 28, 2012 12:26 am    Post subject: Re: can you help me understand pipe and xargs? [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

At Fri, 27 Jan 2012 13:25:32 -0800 (PST) www wrote:

>
> Hi,
>
> I understand pipe using "|": the output of one command becomes the
> input of the second command, for example:
>
> ls | grep abc
>
> will list file names or directory names which contain "abc". I guess
> the effect is same as: ls *abc*

Actually not really.

>
> The question is why we need xargs in piping. For example, the other
> day I ran into this problem: I have a file with name files.txt
>
> cat files.txt:
> a.txt
> b.txt
> ..
>
> I added two dots in purpose as the last line of files.txt. I want to
> copy all file names listed inside files.txt to one level up. My first
> try and it failed:
>
> $cat files.txt | cp
> cp: missing file operand
> Try `cp --help' for more information.
>
> My second try and it worked:
> $cat files.txt | xargs cp
>
> I don't understand what the difference between the two commands. Is
> that xargs put the three lines into one line so cp command can work?
> So the function of "xargs" is joining all the lines it receives into
> one line?

Not exactly. cp takes its arguments on its command line. cat copies
its input to stdout. When you pipe cat's output to cp, you are sending
the text, line by line to cp stdin. cp does not read from its stdin, so
this data is discarded and cp gets nothing on its command line, so it
complains. stdin IS NOT the same as the command line. Yes, xargs takes
its input stream and creates a list of command line arguments from each
of the lines it reads (default behaviour -- read the man page
*carefully* for other behaviours). It does this in batches to limit the
size of the created command line, since there are limits to the size of
the command line the shell can handle.

In your case you could have done:

cp `cat files.txt|tr '\n' ' '`

I'll leave you with the 'homework' assignment of reading the relavant
man pages (man bash and man tr) to figure out just what is going on there.

>
> Thank you very much.
>
>

--
Robert Heller -- 978-544-6933 / heller@deepsoft.com
Deepwoods Software -- http://www.deepsoft.com/
() ascii ribbon campaign -- against html e-mail
/\ www.asciiribbon.org -- against proprietary attachments
Back to top
Robert Heller
External


Since: Sep 13, 2006
Posts: 311



PostPosted: Sat Jan 28, 2012 12:26 am    Post subject: Re: can you help me understand pipe and xargs? [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

At Fri, 27 Jan 2012 13:52:51 -0800 (PST) www wrote:

>
> Just one more example to show my question:
> $cat a.txt
> goo d
> bad g
>
> A.
> $cat a.txt | tac //works great, prints out the lines inside a.txt
> backwards(last line becomes the first line)
> bad g
> goo d
>
> B.
> cat a.txt | echo //does not work ???
>
> C.
> cat a.txt | xargs echo //works, print out the content to screen
>
> D.
> $cat a.txt | xargs tac //does not work ???
> tac: cannot open `goo' for reading: No such file or directory
> tac: cannot open `d' for reading: No such file or directory
> tac: cannot open `bad' for reading: No such file or directory
> tac: cannot open `g' for reading: No such file or directory

stdin is not the command line. The command line is not stdin. It is
rather hard to make apple pie from oranges. And you can't squeeze
orange juice from apples...

>

--
Robert Heller -- 978-544-6933 / heller@deepsoft.com
Deepwoods Software -- http://www.deepsoft.com/
() ascii ribbon campaign -- against html e-mail
/\ www.asciiribbon.org -- against proprietary attachments
Back to top
www
External


Since: Jan 27, 2012
Posts: 10



PostPosted: Sat Jan 28, 2012 2:27 pm    Post subject: Re: can you help me understand pipe and xargs? [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

(This is the reply to all posts responding to me. I am using Google,
all posts are flat now. Unlike newsgroup in Thunderbird, the reply to
a post is put underneath of that post.)

Thank you very much for all the insightful instruction. I greatly
appreciate them.

One gentleman earlier pointed out that to achieve my purpose I could
use this command: cp `cat files.txt`

I tried on my Red Hat Linux and it works fine. Then I tried on cygwin
on my Windows computer, to my surprise, it failed. Generally, my
experience with the two(linux and cygwin) are that they are very much
similar. Now, for the first time, I found that one command works here
and not there. The failure on cygwin seems related to the homework
Robert has assigned to me:
$ cp `cat files.txt` //not working on cygwin;
cp: target `..\r' is not a directory

I created files.txt by using vi on cygwin, it seems there is a hidden
character at the end of two dots. So I should remove that hidden
character. Sorry, I cannot read well Robert's command after the part
\n: cp `cat files.txt|tr '\n' ' '`

I think the command is letting tr delete new line symbol -- the char
of '\n', after receiving three lines from the first action(cat
files.txt). If after the letter n, it is three ' and one back tick,
this command does not work at all.
Back to top
Gary Johnson
External


Since: Jan 29, 2012
Posts: 1



PostPosted: Sun Jan 29, 2012 1:10 am    Post subject: Re: can you help me understand pipe and xargs? [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

www wrote:
> (This is the reply to all posts responding to me. I am using Google,
> all posts are flat now. Unlike newsgroup in Thunderbird, the reply to
> a post is put underneath of that post.)
>
> Thank you very much for all the insightful instruction. I greatly
> appreciate them.
>
> One gentleman earlier pointed out that to achieve my purpose I could
> use this command: cp `cat files.txt`
>
> I tried on my Red Hat Linux and it works fine. Then I tried on cygwin
> on my Windows computer, to my surprise, it failed. Generally, my
> experience with the two(linux and cygwin) are that they are very much
> similar. Now, for the first time, I found that one command works here
> and not there. The failure on cygwin seems related to the homework
> Robert has assigned to me:
> $ cp `cat files.txt` //not working on cygwin;
> cp: target `..\r' is not a directory
>
> I created files.txt by using vi on cygwin, it seems there is a hidden
> character at the end of two dots. So I should remove that hidden
> character. Sorry, I cannot read well Robert's command after the part
> \n: cp `cat files.txt|tr '\n' ' '`

Most Cygwin commands require text files to use only line feeds (LF) as
line terminators, not the carriage return (CR) and line feed sequence
used as line terminators or separators on Windows. When lines ending in
CRLF are read by these Cygwin programs, the programs see the CR as part
of the line, causing various problems as you have discovered.

Vi (actually Vim on Cygwin) can create files with either Unix-style LF
line endings or DOS/Windows-style CRLF line endings. You can force Vim
to save the file with LF line endings by executing this before saving
the file:

:set ff=unix

You can also tell Vim to always use LF line endings by default by
putting this in your ~/.vimrc:

set fileformats=unix,dos

See

:help fileformat
:help fileformats

That said, if you were really using Cygwin's vi and not a Windows vi to
create you files.txt, that file should have been created with LF line
endings because 'fileformates' defaults to "unix,dos" on Cygwin.

> I think the command is letting tr delete new line symbol -- the char
> of '\n', after receiving three lines from the first action(cat
> files.txt). If after the letter n, it is three ' and one back tick,
> this command does not work at all.

If your files.txt has CRLF line terminators, then the CRs will be
considered by cp to be part of the file names. Fix that problem and the
tr command should work.

--
Gary Johnson
Back to top
www
External


Since: Jan 27, 2012
Posts: 10



PostPosted: Sun Jan 29, 2012 1:27 am    Post subject: Re: can you help me understand pipe and xargs? [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

Gary:

Thank you very much for pointing me to a new area: line feed(LF) --
unix style and carriage return line feed(CRLF) - dos style for line
terminator in a file. Honestly, I have never spent time to learn this
thing. Now, it seems a good time for me to chew on it. I am glad and
fortunate enough for doing so by your guidance.

You are absolutely correct. I was not using vi which comes together
with Cygwin. I forgot to choose vi when installing Cygwin. Then, later
I downloaded Vim from vim.org. Now, its default is dos style. Within
the vi editor, the command ":set ff? ffs?" shows the following result:
fileformat=dos
fileformats=dos,unix
So by the order, dos is first, ahead of unix style.

Within the editor, ":set ff=unix" is good for current buffer. In
_vimrc file(I am in Windows computer), I added the following line
*after the line "set nocompatible"*:

:set fileformats=unix,dos
or
:set ffs=unix,dos

Now, back to the original topic, this command works: cp `cat
files.txt` (files.txt was created with vi with fileformats of unix
now)

However, I need more time to finish the 'homework' Robert has assigned
to me: cp 'cat files.txt|tr '\n'''` (after n, three ' then one `, as
what I read, or whatever works). Now I have much better understanding
of his command: he is deleting or translating the line
terminator(whatever the OS is and the corresponding default
fileformats), so with that longer command, when files.txt is line
terminator is dos tyle(CRLF), his command will even work, I guess.
Back to top
Robert Heller
External


Since: Sep 13, 2006
Posts: 311



PostPosted: Sun Jan 29, 2012 3:28 am    Post subject: Re: can you help me understand pipe and xargs? [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

At Sat, 28 Jan 2012 09:27:39 -0800 (PST) www wrote:

>
> (This is the reply to all posts responding to me. I am using Google,
> all posts are flat now. Unlike newsgroup in Thunderbird, the reply to
> a post is put underneath of that post.)
>
> Thank you very much for all the insightful instruction. I greatly
> appreciate them.
>
> One gentleman earlier pointed out that to achieve my purpose I could
> use this command: cp `cat files.txt`
>
> I tried on my Red Hat Linux and it works fine. Then I tried on cygwin
> on my Windows computer, to my surprise, it failed. Generally, my
> experience with the two(linux and cygwin) are that they are very much
> similar. Now, for the first time, I found that one command works here
> and not there. The failure on cygwin seems related to the homework
> Robert has assigned to me:
> $ cp `cat files.txt` //not working on cygwin;
> cp: target `..\r' is not a directory
>
> I created files.txt by using vi on cygwin, it seems there is a hidden
> character at the end of two dots. So I should remove that hidden
> character. Sorry, I cannot read well Robert's command after the part

DOS newfiles: \r\n -- bad news.

> \n: cp `cat files.txt|tr '\n' ' '`

it is three 'words':

tr (the translate command -- translates single characters in files or
pipes)
'\n' The is the UNIX newline character, quoted with apostrophes -- a
single character, the ASCII Linefeed character
' ' A single space in quotes (apostrophes).

the command

tr '\n' ' '

translates newlines to spaces.

Thus the command

cat files.txt|tr '\n' ' '

translates translates the contents of files.txt from lines to words.
For example if file.txt contains the lines:

a.file
b.file
c.file

the output of the above is

a.file b.file c.file

It also works to do

tr '\n' ' ' <files.txt

>
> I think the command is letting tr delete new line symbol -- the char
> of '\n', after receiving three lines from the first action(cat
> files.txt). If after the letter n, it is three ' and one back tick,
> this command does not work at all.

It does not 'delete' the newline, it replaces it with a space. You need
to preserve the spaces in the command line. They are important. Spaces
in command lines are *always* important and always have special meaning
(unless they are quoted). Spaces in file names are a pain in the rear
when working at the command line level.
>

--
Robert Heller -- 978-544-6933 / heller@deepsoft.com
Deepwoods Software -- http://www.deepsoft.com/
() ascii ribbon campaign -- against html e-mail
/\ www.asciiribbon.org -- against proprietary attachments
Back to top
www
External


Since: Jan 27, 2012
Posts: 10



PostPosted: Sun Jan 29, 2012 3:14 pm    Post subject: Re: can you help me understand pipe and xargs? [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

As I think more, I feel I have more questions in my mind:

Question 1. the following two commands work with pipe and without
xargs:

cat files.txt | tac //tac receives stdin -- not arguments -- from the
action of "cat files.txt" == this command "tac < files.txt". OK, I
think I can accept it
ls | grep abc //?

grep does not receive stdin, in my understanding. For example, with
this command, "grep this_word f.txt", I would say "this_word f.txt"
are two arguments to the command "grep". So, why it works here without
xargs. Actually, with xargs, it failes. In addition, if I just type
"grep" and hit enter, the shell immediately gives this message, not
waiting for input from my keyboard:
$ grep
Usage: grep [OPTION]... PATTERN [FILE]...
Try `grep --help' for more information.

Question 1B. Another related question is, when I type "tac" and hit
enter, it is the blank line, ok, it is waiting for the input, so I
type something, hit enter, it is another blank line, repeat the
action, it is always another blank line. Control-C terminates it, but
no any lines printed out to the screen. I know "tac < file.txt" works,
but how to understand this.

Question 2. More or less, the two commands are equivalent, in my
understaning:

cp `cat files.txt`
cat files.txt | xargs cp

$ cat files.txt
a.txt
b.txt
...

So, both commands above copy two files to one level up. Now, I run
into this puzzle:
$ cat a.txt
temp

cd `cat a.txt` //this command successfully cd into the subdir of
"temp"
$ cat a.txt | xargs cd //failed on both Red Hat Linux and Cygwin
xargs: cd: No such file or directory

So the shell cannot find this command "cd"?! I typed "which cd" and
indeed, it is not in the path. I don't understand.

Thank you very much for helping me.
Back to top
framefritti
External


Since: Jan 27, 2012
Posts: 3



PostPosted: Sun Jan 29, 2012 3:54 pm    Post subject: Re: can you help me understand pipe and xargs? [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

Il giorno domenica 29 gennaio 2012 19:14:42 UTC+1, www ha scritto:
> As I think more, I feel I have more questions in my mind:
>
> Question 1. the following two commands work with pipe and without
> xargs:
>
> cat files.txt | tac //tac receives stdin -- not arguments -- from the
> action of "cat files.txt" == this command "tac < files.txt". OK, I
> think I can accept it
> ls | grep abc //?
>
> grep does not receive stdin, in my understanding. For example, with
> this command, "grep this_word f.txt", I would say "this_word f.txt"
> are two arguments to the command "grep".

grep reads the string to be searched from command line, but if no filename is given, it searches the string in the standard input. For example, the two following ways of calling grep are (more or less) equivalent

grep search-me file.txt # grep reads file.txt

grep search-me < file.txt # since no filename is given,
# grep reads stdin (standard input)
# that in this special case contains
# the content of file.txt


> So, why it works here without
> xargs. Actually, with xargs, it failes. In addition, if I just type
> "grep" and hit enter, the shell immediately gives this message, not
> waiting for input from my keyboard:
> $ grep

Yes, because the pattern is missing, so grep does not what to search for. Try to run with the pattern only

grep search-me

> Usage: grep [OPTION]... PATTERN [FILE]...
> Try `grep --help' for more information.
>
(omiss)

> cd `cat a.txt` //this command successfully cd into the subdir of
> "temp"
> $ cat a.txt | xargs cd //failed on both Red Hat Linux and Cygwin
> xargs: cd: No such file or directory
>
> So the shell cannot find this command "cd"?! I typed "which cd" and
> indeed, it is not in the path. I don't understand.
>

Smile ...I remember when (many years ago, at the beginning of my Unix experience) I felt in this trapdoor too... Smile

You see, cd is not an external program, but a built-in of the shell. The reason for this is quite "deep" and it depends on how the shell executes the external commands. Very briefly (and oversimplifying a bit), when you type "ls" the shell

* creates a "clone" (a child process) of itself by calling fork(). I do not know your experience with Unix API, just in case I tell you that when you call fork() the system creates a more or less identical copy of the program (better, the process) in execution and both processes run in parallel. The child process (the new one) knows that it is the child because fork() return 0, while in the parent process (the old one) fork() returns the process number of the child.

* the child process calls exec() in order to have its executable code replaced with the executable code of "ls"

* "ls" runs and does what it has to do

What all this have in common with the "cd" problem? Just think what would happen if "cd" was an external command

* the shell would fork() itself

* the child process would change its working directory...

* ...but the parent (the shell you are using) would still have the same working directory, since the child changed its own, not the directory of the parent

Therefore, because of this mechanism, "cd" cannot implemented as an external program, but it must always be a built-in. This explains why xargs cannot find it.


> Thank you very much for helping me.
Back to top
Robert Heller
External


Since: Sep 13, 2006
Posts: 311



PostPosted: Sun Jan 29, 2012 8:38 pm    Post subject: Re: can you help me understand pipe and xargs? [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

At Sun, 29 Jan 2012 10:14:42 -0800 (PST) www wrote:

>
> As I think more, I feel I have more questions in my mind:
>
> Question 1. the following two commands work with pipe and without
> xargs:
>
> cat files.txt | tac //tac receives stdin -- not arguments -- from the
> action of "cat files.txt" == this command "tac < files.txt". OK, I
> think I can accept it
> ls | grep abc //?
>
> grep does not receive stdin, in my understanding. For example, with
> this command, "grep this_word f.txt", I would say "this_word f.txt"
> are two arguments to the command "grep". So, why it works here without
> xargs. Actually, with xargs, it failes. In addition, if I just type
> "grep" and hit enter, the shell immediately gives this message, not
> waiting for input from my keyboard:
> $ grep
> Usage: grep [OPTION]... PATTERN [FILE]...
> Try `grep --help' for more information.
>
> Question 1B. Another related question is, when I type "tac" and hit
> enter, it is the blank line, ok, it is waiting for the input, so I
> type something, hit enter, it is another blank line, repeat the
> action, it is always another blank line. Control-C terminates it, but
> no any lines printed out to the screen. I know "tac < file.txt" works,
> but how to understand this.
>
> Question 2. More or less, the two commands are equivalent, in my
> understaning:
>
> cp `cat files.txt`
> cat files.txt | xargs cp
>
> $ cat files.txt
> a.txt
> b.txt
> ..
>

UNIX 101: From the point of view of programs, *everything* is a file,
including stdin, stdout, and stderr. Also pipes and sockets. The shell
creates pipes on demand (whenever the pipe operator (|) shows up).

So, *any* program that opens and reads a file can also read a pipe or
the terminal keyboard. Any program the opens and writes to a file, can
also write to a pipe or a terminal screen.

grep takes its pattern from the command line. It can take files to
match on the command line *OR* it can just match against the stream
available on stdin.

cat concatenates (writes the contents to stdout, one by one) the files
listed on its command line. It can also write the contents of stdin to
stdout. And can concatenate one or more files with stdin (see below).

*Some* commands take the special filename '-' to indicate stdin. This
would be for commands that don't naturally read or write to stdin/stdout
in the absense of a file to read or write from (tar is an example of
this). cat will also do this:

cat files.txt - >filesmore.txt
another.txt
^D

This concatenates some input from stdin (the terminal keyboard) to the
end of a file, and saves it in another file.

> So, both commands above copy two files to one level up. Now, I run
> into this puzzle:
> $ cat a.txt
> temp
>
> cd `cat a.txt` //this command successfully cd into the subdir of
> "temp"
> $ cat a.txt | xargs cd //failed on both Red Hat Linux and Cygwin
> xargs: cd: No such file or directory
>
> So the shell cannot find this command "cd"?! I typed "which cd" and
> indeed, it is not in the path. I don't understand.

cd is a built-in command in the shell. There is no /bin/cd (or
/usr/bin/cd) program file. xargs attempts to fork() and execv() 'cd'
-- this will fail. cd is a local context side effect command --
fork'ing a fresh process (new local context) and then doing a cd only
effects the newly fork'ed process. The parent process is not affected.
It only makes sense to do a cd in a child process if that is part of a
larger program, like inside a script -- that is, if the program execv'ed
after the fork is /bin/bash and the script passed to /bin/bash does the
cd which is (presumably) followed by additional bash commands that need
to be run in the specified directory.

Consider this:

cat >test.sh
#!/bin/bash
cd $1
echo "I am here $CWD"
^D

chmod +x test.sh

cat a.txt | xargs -L 1 `pwd`/test.sh

>
> Thank you very much for helping me.


--
Robert Heller -- 978-544-6933 / heller@deepsoft.com
Deepwoods Software -- http://www.deepsoft.com/
() ascii ribbon campaign -- against html e-mail
/\ www.asciiribbon.org -- against proprietary attachments
Back to top
Richard Kettlewell
External


Since: Feb 19, 2005
Posts: 217



PostPosted: Sun Jan 29, 2012 9:10 pm    Post subject: Re: can you help me understand pipe and xargs? [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)

www writes:
> Question 1. the following two commands work with pipe and without
> xargs:
>
> cat files.txt | tac //tac receives stdin -- not arguments -- from the
> action of "cat files.txt" == this command "tac < files.txt". OK, I
> think I can accept it
> ls | grep abc //?
>
> grep does not receive stdin, in my understanding. For example, with
> this command, "grep this_word f.txt", I would say "this_word f.txt"
> are two arguments to the command "grep". So, why it works here without
> xargs. Actually, with xargs, it failes. In addition, if I just type
> "grep" and hit enter, the shell immediately gives this message, not
> waiting for input from my keyboard:
> $ grep
> Usage: grep [OPTION]... PATTERN [FILE]...
> Try `grep --help' for more information.

The PATTERN argument is mandatory. "grep something" will wait for input.

> Question 1B. Another related question is, when I type "tac" and hit
> enter, it is the blank line, ok, it is waiting for the input, so I
> type something, hit enter, it is another blank line, repeat the
> action, it is always another blank line. Control-C terminates it, but
> no any lines printed out to the screen. I know "tac < file.txt" works,
> but how to understand this.

^C interrupts the process. Use ^D to finish the terminal input.

> So, both commands above copy two files to one level up. Now, I run
> into this puzzle:
> $ cat a.txt
> temp
>
> cd `cat a.txt` //this command successfully cd into the subdir of
> "temp"
> $ cat a.txt | xargs cd //failed on both Red Hat Linux and Cygwin
> xargs: cd: No such file or directory
>
> So the shell cannot find this command "cd"?! I typed "which cd" and
> indeed, it is not in the path. I don't understand.

cd is built into the shell and (in typical Linux systems) does not exist
as a separate executable, so xargs can't find it.

This is actually a spec violation. However fixing it wouldn't make any
useful difference, at least it this case: only builtins can modify the
shell's working directory.

--
http://www.greenend.org.uk/rjk/
Back to top
Display posts from previous:   
Post new topic   General Reply to Topic (not reply to a specific post)    Forums Home -> Genreal Discussions All times are: Eastern Time (US & Canada)
Page 1 of 1

 
You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum