|
Home > Archive > Data Storage > November 2007 > inode usage / file
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
inode usage / file
|
|
|
| Hi,
I have a 1.4 TB Volume exported over NFS, accessed from a Linux
NFS(v3) client. I have to fill all indoes on this volume. One way of
doing it is to create so many files which is time consuming . One more
option to use is Snapmirror/Replication technologies to create this
Data set. Anything else more efficient than either of this ?
Can there be more than one inode per file ? I mean if I create a file
of 100gb for example, will it utilize only one inode or inode +
indirect inodes = total inode count ? And How to see per file inode
usage on linux ?
Please note I am talking about Netapp Wafl FS here.
I am trying my best to find out the answers, if anybody can help cut
short the time, will be great full.
thanks
Parinay
| |
| Dan Rumney 2007-11-02, 1:15 pm |
| See http://en.wikipedia.org/wiki/Inode for information about inodes.
As noted there, each file maps on to a single inode.
Therefore, if you want to use up all the inodes, just create a script
which writes many small files. There's nothing particularly
inefficient about this. It'll simply take some time to complete due to
simple fact that it takes time to write things
| |
| Bakul Shah 2007-11-02, 1:15 pm |
| zorba wrote:
> I have a 1.4 TB Volume exported over NFS, accessed from a Linux
> NFS(v3) client. I have to fill all indoes on this volume. One way of
> doing it is to create so many files which is time consuming . One more
> option to use is Snapmirror/Replication technologies to create this
> Data set. Anything else more efficient than either of this ?
Write a script. In /bin/sh:
x=0; while touch $x; do x=$(($x + 1)); done
> Can there be more than one inode per file ? I mean if I create a file
> of 100gb for example, will it utilize only one inode or inode +
> indirect inodes = total inode count ? And How to see per file inode
> usage on linux ?
One inode is used per file. Use
df -i
to see the inode count.
> Please note I am talking about Netapp Wafl FS here.
>
> I am trying my best to find out the answers, if anybody can help cut
> short the time, will be great full.
I sense a lot of confusion. I am not even sure if it is inodes
you want. What you wrote seems to make more sense if the word
"inode" is replaced with the word "block". If I were you I'd focus
on learning the basic concepts as that will save more time and
pain in the long run.
| |
|
| On Nov 2, 9:51 pm, Bakul Shah <use...@bitblocks.com> wrote:
> zorba wrote:
>
> Write a script. In /bin/sh:
>
> x=0; while touch $x; do x=$(($x + 1)); done
>
>
> One inode is used per file. Use
>
> df -i
>
> to see the inode count.
>
>
>
> I sense a lot of confusion. I am not even sure if it is inodes
> you want. What you wrote seems to make more sense if the word
> "inode" is replaced with the word "block". If I were you I'd focus
> on learning the basic concepts as that will save more time and
> pain in the long run.
Agreed. Thanks a lot.
| |
| Faeandar 2007-11-03, 1:14 am |
| On Fri, 02 Nov 2007 05:02:28 -0000, zorba <parinay@gmail.com> wrote:
>Hi,
>I have a 1.4 TB Volume exported over NFS, accessed from a Linux
>NFS(v3) client. I have to fill all indoes on this volume. One way of
>doing it is to create so many files which is time consuming . One more
>option to use is Snapmirror/Replication technologies to create this
>Data set. Anything else more efficient than either of this ?
>Can there be more than one inode per file ? I mean if I create a file
>of 100gb for example, will it utilize only one inode or inode +
>indirect inodes = total inode count ? And How to see per file inode
>usage on linux ?
>
>Please note I am talking about Netapp Wafl FS here.
>
>I am trying my best to find out the answers, if anybody can help cut
>short the time, will be great full.
>
>thanks
>Parinay
I'm curious why you need to use all the inodes? One posters
distinction between blocks and inodes may have shed some light.
Are you trying to "burn-in" a filer or volume? You can use 'hammer'
if that is the case. It's a built-in command in Ontap.
~F
| |
| the wharf rat 2007-11-05, 1:15 pm |
| In article <1193979748.393165.23680@i38g2000prf.googlegroups.com>,
zorba <parinay@gmail.com> wrote:
>Hi,
>I have a 1.4 TB Volume exported over NFS, accessed from a Linux
>NFS(v3) client. I have to fill all indoes on this volume. One way of
>doing it is to create so many files which is time consuming . One more
export foo=1
while (`true`)
do
ln /firstfile /firstfile.${foo}
if [ "$?" = "1" ]
then
break
fi
foo=`expr foo + 1`
done
Wafl shoudn't have any problem with the pathologically large
directory you'll end up with...
| |
|
| On Nov 5, 11:10 am, w...@panix.com (the wharf rat) wrote:
> In article <1193979748.393165.23...@i38g2000prf.googlegroups.com>,
>
> zorba <pari...@gmail.com> wrote:
>
> export foo=1
> while (`true`)
> do
> ln /firstfile /firstfile.${foo}
> if [ "$?" = "1" ]
> then
> break
> fi
> foo=`expr foo + 1`
> done
>
> Wafl shoudn't have any problem with the pathologically large
> directory you'll end up with...
"ln" command does not create new inodes; it just creates new directory
entries. "ln -s" (symbolic link) can create new inodes.
Wafl does have limitation on number of entries in a directory, but it
is big.
If the desired action is to fill all the blocks, "ndmpcopy" can be
used to copy data between volumes (or copy data within a volume).
However, one may use up all inodes before all blocks are used. Then,
"maxfiles" can be used to add more inodes to a volume.
| |
| Edwin Cooke 2007-11-09, 7:12 pm |
| In message <1193979748.393165.23680@i38g2000prf.googlegroups.com>,
zorba <parinay(at)gmail.com> wrote:
>
> Hi,
> I have a 1.4 TB Volume exported over NFS, accessed from a Linux
> NFS(v3) client. I have to fill all indoes on this volume. One way of
> doing it is to create so many files which is time consuming . One more
> option to use is Snapmirror/Replication technologies to create this
> Data set. Anything else more efficient than either of this ?
> Can there be more than one inode per file ? I mean if I create a file
> of 100gb for example, will it utilize only one inode or inode +
> indirect inodes = total inode count ? And How to see per file inode
> usage on linux ?
>
> Please note I am talking about Netapp Wafl FS here.
and in message <fgnprf$rel$1@reader1.panix.com>,
"the wharf rat" <wrat(at)panix.com> replied
|
| >Hi,
| >I have a 1.4 TB Volume exported over NFS, accessed from a Linux
| >NFS(v3) client. I have to fill all indoes on this volume. One way of
| >doing it is to create so many files which is time consuming . One more
|
| [suggested script omitted]
|
| Wafl shoudn't have any problem with the pathologically large
| directory you'll end up with...
Wharf Rat, you must be joking, right?
WAFL's performance will degrade for very large dirctories.
In message <472b5587$0$79949$742ec2ed@news.sonic.net>,
Bakul Shah <usenet(at)bitblocks.com> wrote
}
} zorba wrote:
}
} > I have a 1.4 TB Volume exported over NFS, accessed from a Linux
} > NFS(v3) client. I have to fill all indoes on this volume. One way of
} > doing it is to create so many files which is time consuming . One more
} > option to use is Snapmirror/Replication technologies to create this
} > Data set. Anything else more efficient than either of this ?
}
} Write a script. In /bin/sh:
}
} x=0; while touch $x; do x=$(($x + 1)); done
}
} > Can there be more than one inode per file ? I mean if I create a file
} > of 100gb for example, will it utilize only one inode or inode +
} > indirect inodes = total inode count ? And How to see per file inode
} > usage on linux ?
}
} One inode is used per file. Use
}
} df -i
}
} to see the inode count.
}
} > Please note I am talking about Netapp Wafl FS here.
} >
} > I am trying my best to find out the answers, if anybody can help cut
} > short the time, will be great full.
}
} I sense a lot of confusion. I am not even sure if it is inodes
} you want. What you wrote seems to make more sense if the word
} "inode" is replaced with the word "block". If I were you I'd focus
} on learning the basic concepts as that will save more time and
} pain in the long run.
NetApp's WAFL, the "write-anywhere file layout", is enough different from
other filesystems (such as ufs, ntfs, or ext2fs) that advice which makes
sense for those filesystems does *not* necessarily apply to WAFL.
By default, NetApp's WAFL filesystem allocates one inode (to store one
file) for every 32K bytes of disk space in a volume.
The "df -i" command will work on a Linux client, but some NFS clients
may not support it. On the other hand, the NetApp filer itself (through
the Data ONTAP command-line interface) does support "df -i".
As Bakul has observed, there does seem to be confusion here.
Zorba, it is not clear what you mean by "I have to fill all indoes"
[presumably "inodes"?]. Does that mean that you want to create a
number of files equal to the "maxfiles" value? (The per-volume
value reported by the filer for the "maxfiles" command output.)
Are you running some kind of performance or NFS stress test?
Did you make some intentional adjustment of "maxfiles"?
While the touch command will create a zero-length file (thus
allocating an inode), this is not a typical usage pattern, so
any results from the test may not apply to real-life operation.
--
Edwin
| |
| the wharf rat 2007-11-10, 1:14 am |
| In article <1194643786.726186.103640@k35g2000prh.googlegroups.com>,
hsing <hsingy101@gmail.com> wrote:
>
>"ln" command does not create new inodes; it just creates new directory
>entries. "ln -s" (symbolic link) can create new inodes.
>
Ok, so I forgot the -s. I'll turn in my root password :-)
>If the desired action is to fill all the blocks, "ndmpcopy" can be
>used to copy data between volumes (or copy data within a volume).
I think the guy wanted to avoid allocating actual data blocks
because of time and space limitations. ln will cause any number of inodes
to point to the same data block so it might do what he wants.
| |
| the wharf rat 2007-11-10, 1:14 am |
| In article <ecoo_20071109_141011-TYS@stratagy.com>,
Edwin Cooke <ecoo@list.stratagy.com> wrote:
>
>Wharf Rat, you must be joking, right?
>
>WAFL's performance will degrade for very large dirctories.
>
Not nearly as much as UFS or EXT2 in my experience.
| |
|
| On Nov 10, 3:10 am, "Edwin Cooke" <e...@list.stratagy.com> wrote:
> In message <1193979748.393165.23...@i38g2000prf.googlegroups.com>,
>
> zorba <parinay(at)gmail.com> wrote:
>
>
>
> and in message <fgnprf$re...@reader1.panix.com>,
> "the wharf rat" <wrat(at)panix.com> replied
> |
> | >Hi,
> | >I have a 1.4 TB Volume exported over NFS, accessed from a Linux
> | >NFS(v3) client. I have to fill all indoes on this volume. One way of
> | >doing it is to create so many files which is time consuming . One more
> |
> | [suggested script omitted]
> |
> | Wafl shoudn't have any problem with the pathologically large
> | directory you'll end up with...
>
> Wharf Rat, you must be joking, right?
>
> WAFL's performance will degrade for very large dirctories.
>
> In message <472b5587$0$79949$742ec...@news.sonic.net>,
> Bakul Shah <usenet(at)bitblocks.com> wrote
>
>
>
> }
> } zorba wrote:
> }
> } > I have a 1.4 TB Volume exported over NFS, accessed from a Linux
> } > NFS(v3) client. I have to fill all indoes on this volume. One way of
> } > doing it is to create so many files which is time consuming . One more
> } > option to use is Snapmirror/Replication technologies to create this
> } > Data set. Anything else more efficient than either of this ?
> }
> } Write a script. In /bin/sh:
> }
> } x=0; while touch $x; do x=$(($x + 1)); done
> }
> } > Can there be more than one inode per file ? I mean if I create a file
> } > of 100gb for example, will it utilize only one inode or inode +
> } > indirect inodes = total inode count ? And How to see per file inode
> } > usage on linux ?
> }
> } One inode is used per file. Use
> }
> } df -i
> }
> } to see the inode count.
> }
> } > Please note I am talking about Netapp Wafl FS here.
> } >
> } > I am trying my best to find out the answers, if anybody can help cut
> } > short the time, will be great full.
> }
> } I sense a lot of confusion. I am not even sure if it is inodes
> } you want. What you wrote seems to make more sense if the word
> } "inode" is replaced with the word "block". If I were you I'd focus
> } on learning the basic concepts as that will save more time and
> } pain in the long run.
>
> NetApp's WAFL, the "write-anywhere file layout", is enough different from
> other filesystems (such as ufs, ntfs, or ext2fs) that advice which makes
> sense for those filesystems does *not* necessarily apply to WAFL.
>
> By default, NetApp's WAFL filesystem allocates one inode (to store one
> file) for every 32K bytes of disk space in a volume.
>
> The "df -i" command will work on a Linux client, but some NFS clients
> may not support it. On the other hand, the NetApp filer itself (through
> the Data ONTAP command-line interface) does support "df -i".
>
> As Bakul has observed, there does seem to be confusion here.
>
> Zorba, it is not clear what you mean by "I have to fill all indoes"
> [presumably "inodes"?]. Does that mean that you want to create a
> number of files equal to the "maxfiles" value? (The per-volume
> value reported by the filer for the "maxfiles" command output.)
> Are you running some kind of performance or NFS stress test?
> Did you make some intentional adjustment of "maxfiles"?
>
> While the touch command will create a zero-length file (thus
> allocating an inode), this is not a typical usage pattern, so
> any results from the test may not apply to real-life operation.
>
> --
> Edwin
Edwin,
--I mean inodes here ( esp after bakul's comment, I have really went
back to Morris Bach )
--I had modified default maxfiles count to what I needed
--Creating a soft link to a single data block seems to be an efficient
way, for now
--Scenario is to utilize maximum available inodes and then initiating
snapmirror resync. The filer should not panic
--Utilizing inodes is over NFS
> By default, NetApp's WAFL filesystem allocates one inode (to store one
> file) for every 32K bytes of disk space in a volume
For file disk space more than 32K, how does it work ?
best regards
Zorba
|
|
|
|
|