f



Re: raw vs. cooked files under linux #4

I guess almost all of us are convinced at this point of the performance
improvement
from using raw devices. But it would be great if we could get a benchmark
report
with a couple of Filesystems, specially on Linux, just in case we really
need to use Filesystems.
(JFS, Ext3, XFS , ReiserFS, ... )
I am a fan of XFS on Linux, because of its speed/stability/reliability
against the other
filesystems. I would be glad to see if XFS is better for database-OLTP
performance or not.

Could someone point us to a benchmark report  of filesystems/raw devices  ?

Best Regards

----- Original Message -----
From: "Art S. Kagel" <kagel@bloomberg.net>
To: <informix-list@iiug.org>
Sent: Wednesday, May 11, 2005 1:30 PM
Subject: Re: raw vs. cooked files under linux


> Heinz Weitkamp wrote:
> > Hi all,
>
> Go RAW all the way.  Contrary to the Goob's understanding my own testing
> shows RAW devices to be 15-25% faster than COOKED devices which are in
turn
> 10-15% faster than filesystem files, even on the best filesystem
> implementations that adds up to a 25% improvement.  Well worth any minor
> inconveniences in my book.  I have not yet tested Linux RAW devices versus
> the various filesystems that Linux directly supports, but early reports
when
> Linux RAW devices were first rolled out showed similar results.  The
> performance gain was one reason why Linus Torvalds was convinced finally
to
> include RAW devices in Linux, a development he had resisted for years.
The
> other major reason being the need to provide high speed low level IO
> capability to software like IDS that manages its own disk space.
>
> I also don't know why Data Goob thinks that you can backup cooked files or
> devices for IDS any differently than you can raw devices.  An archive
taken
>   without using ontape or onbar is known as an 'external' archive.  In all
> three cases you can use ontape or onbar to perform online backups.  In all
> three cases if you want to produce an external archive successfully (read
> "be able to restore the data later") you have to perform the archive while
> the engine is not modifying the disk contents.  That means one of three
> scenarios 1, 2a, or 2b:
>
> 1) Shutdown the engine and use dd, tar, Legato Networker, or whatever to
> copy the chunk files to tape/CD/DVD/whatever.
> 2) Run onmode -c block to force a hard checkpoint and block the server
from
> modifying anything until you're backup is complete.  Then you either:
>     a) Back up the chunks as in 1) or
>     b) Break mirrors, onmode -c unblock, backup the mirrors, recreate the
> mirrors.
>
> UNDER NO CIRCUMSTANCES can you successfully archive the underlying chunk
> files - whether they are RAW, COOKED, or files - while the IDS engine is
> actively processing transactions!  For an OLTP or OLTP-like server, you
> could not reliably successfully restore the resulting archive.  AN
> unreliable archive is no archive at all!
>
> Sure if you're running a DW server and nothing's been modified in a week
and
> there was a recent hard checkpoint (or you forced one) you'll probably
luck
> out and get a usable external backup without blocking or shutting down the
> engine.
>
> Art S. Kagel
>
> > i am an linux newbie.
> > I must shift our Databases from SCO Unix  (IDS 7.31 UD4) to SuSE Linux
> > Enterprise Edition 8.1 (2.4.21-190-smp glibc 2.2.5) or (8.2) or 9 and
SuSE
> > Linux Professional 9.3 (2.6.11.4 glibc 2.3.4) (IDS 7.31 UD8).
> >
> > Later on i want to upgrade (in place) to IDS 10.0.
> > Which Version of SuSE Linux Enterprise Edition must I use (8.? , 9 ..)?
> >
> > Under Sco Unix we use raw-devices.
> > Is it better to use cooked files or raw files under these
Linux-Versions?
> > A colleague mentioned, it can be, that later releases of Linux don't
support
> > raw-devices. Is that correct?
> >
> > Experiences or comments are welcome.
> > Thanks in advance.
> > (Excuse my poor school-english)
> >
> > Heinz
> >
> > sending to informix-list

sending to informix-list
0
Francisco
5/11/2005 10:26:54 PM
comp.databases.informix 16083 articles. 0 followers. Post Follow

3 Replies
606 Views

Similar Articles

[PageSpeed] 46

You should try Google, a search engine on the internet.  You can
put phrases into it and results come back with page links to different
web sites.  This is but a partial listing for raw vs cooked files and
some other similar phrases:

http://fsbench.netnation.com/

Home of ReiserFS:  http://www.namesys.com/

http://linuxgazette.net/102/piszcz.html

http://www.linux-mag.com/2002-10/jfs_01.html

http://www.yolinux.com/TUTORIALS/LinuxClustersAndFileSystems.html

http://pgmeter.sourceforge.net/pgmeter.pdf

http://kerneltrap.org/node/715

This one is really good:
http://www.quest-pipelines.com/newsletter-v2/linux2.htm

http://librenix.com/?inode=3495

What Legato thinks:
http://www.nocoug.org/download/2003-02/ora_backup.ppt


Francisco Roldan wrote:
> I guess almost all of us are convinced at this point of the performance
> improvement
> from using raw devices. But it would be great if we could get a benchmark
> report
> with a couple of Filesystems, specially on Linux, just in case we really
> need to use Filesystems.
> (JFS, Ext3, XFS , ReiserFS, ... )
> I am a fan of XFS on Linux, because of its speed/stability/reliability
> against the other
> filesystems. I would be glad to see if XFS is better for database-OLTP
> performance or not.
> 
> Could someone point us to a benchmark report  of filesystems/raw devices  ?
> 
> Best Regards
> 
> ----- Original Message -----
> From: "Art S. Kagel" <kagel@bloomberg.net>
> To: <informix-list@iiug.org>
> Sent: Wednesday, May 11, 2005 1:30 PM
> Subject: Re: raw vs. cooked files under linux
> 
> 
> 
>>Heinz Weitkamp wrote:
>>
>>>Hi all,
>>
>>Go RAW all the way.  Contrary to the Goob's understanding my own testing
>>shows RAW devices to be 15-25% faster than COOKED devices which are in
> 
> turn
> 
>>10-15% faster than filesystem files, even on the best filesystem
>>implementations that adds up to a 25% improvement.  Well worth any minor
>>inconveniences in my book.  I have not yet tested Linux RAW devices versus
>>the various filesystems that Linux directly supports, but early reports
> 
> when
> 
>>Linux RAW devices were first rolled out showed similar results.  The
>>performance gain was one reason why Linus Torvalds was convinced finally
> 
> to
> 
>>include RAW devices in Linux, a development he had resisted for years.
> 
> The
> 
>>other major reason being the need to provide high speed low level IO
>>capability to software like IDS that manages its own disk space.
>>
>>I also don't know why Data Goob thinks that you can backup cooked files or
>>devices for IDS any differently than you can raw devices.  An archive
> 
> taken
> 
>>  without using ontape or onbar is known as an 'external' archive.  In all
>>three cases you can use ontape or onbar to perform online backups.  In all
>>three cases if you want to produce an external archive successfully (read
>>"be able to restore the data later") you have to perform the archive while
>>the engine is not modifying the disk contents.  That means one of three
>>scenarios 1, 2a, or 2b:
>>
>>1) Shutdown the engine and use dd, tar, Legato Networker, or whatever to
>>copy the chunk files to tape/CD/DVD/whatever.
>>2) Run onmode -c block to force a hard checkpoint and block the server
> 
> from
> 
>>modifying anything until you're backup is complete.  Then you either:
>>    a) Back up the chunks as in 1) or
>>    b) Break mirrors, onmode -c unblock, backup the mirrors, recreate the
>>mirrors.
>>
>>UNDER NO CIRCUMSTANCES can you successfully archive the underlying chunk
>>files - whether they are RAW, COOKED, or files - while the IDS engine is
>>actively processing transactions!  For an OLTP or OLTP-like server, you
>>could not reliably successfully restore the resulting archive.  AN
>>unreliable archive is no archive at all!
>>
>>Sure if you're running a DW server and nothing's been modified in a week
> 
> and
> 
>>there was a recent hard checkpoint (or you forced one) you'll probably
> 
> luck
> 
>>out and get a usable external backup without blocking or shutting down the
>>engine.
>>
>>Art S. Kagel
>>
>>
>>>i am an linux newbie.
>>>I must shift our Databases from SCO Unix  (IDS 7.31 UD4) to SuSE Linux
>>>Enterprise Edition 8.1 (2.4.21-190-smp glibc 2.2.5) or (8.2) or 9 and
> 
> SuSE
> 
>>>Linux Professional 9.3 (2.6.11.4 glibc 2.3.4) (IDS 7.31 UD8).
>>>
>>>Later on i want to upgrade (in place) to IDS 10.0.
>>>Which Version of SuSE Linux Enterprise Edition must I use (8.? , 9 ..)?
>>>
>>>Under Sco Unix we use raw-devices.
>>>Is it better to use cooked files or raw files under these
> 
> Linux-Versions?
> 
>>>A colleague mentioned, it can be, that later releases of Linux don't
> 
> support
> 
>>>raw-devices. Is that correct?
>>>
>>>Experiences or comments are welcome.
>>>Thanks in advance.
>>>(Excuse my poor school-english)
>>>
>>>Heinz
>>>
>>>sending to informix-list
> 
> 
> sending to informix-list

0
Data
5/12/2005 1:51:05 AM
Francisco Roldan wrote:
> I guess almost all of us are convinced at this point of the performance
> improvement
> from using raw devices. But it would be great if we could get a benchmark
> report
> with a couple of Filesystems, specially on Linux, just in case we really
> need to use Filesystems.
> (JFS, Ext3, XFS , ReiserFS, ... )
> I am a fan of XFS on Linux, because of its speed/stability/reliability
> against the other
> filesystems. I would be glad to see if XFS is better for database-OLTP
> performance or not.
> 
> Could someone point us to a benchmark report  of filesystems/raw devices  ?
> 
> Best Regards
> 
A few years ago running IDS 9.1 on Solaris 7, I found that 2GB chunk initialize 
was 2 to 3 times slower on cooked vs raw chunks.

Row Inserts on cooked chunks were at barely 50% of the speed compared to raw 
chunks.  Currently we are testing a RAID-10 cooked configuration big chunks and 
it still appears slower than the old config with raw devices.  However no test 
times for comparisons were actually recorded.  (Just user complaints)

Bob
0
Bobb
5/12/2005 6:36:41 AM
Francisco Roldan wrote:
> I guess almost all of us are convinced at this point of the performance
> improvement
> from using raw devices. But it would be great if we could get a benchmark
> report
> with a couple of Filesystems, specially on Linux, just in case we really
> need to use Filesystems.
> (JFS, Ext3, XFS , ReiserFS, ... )
> I am a fan of XFS on Linux, because of its speed/stability/reliability
> against the other
> filesystems. I would be glad to see if XFS is better for database-OLTP
> performance or not.
> 
> Could someone point us to a benchmark report  of filesystems/raw devices  ?

Anyone doing this remember to test the COOKED devices and FS files using 
open with O_SYNC to force a physical write wait which is how IDS always 
performs non-RAW writes so that it knows the write was committed to the 
platters before acknowledging the IO as completed.  Otherwise you are 
comparing disk IO speed with memory write speed and the results will not 
hold in production.

Be careful of OS and SAN vendor benchmarks which compare RAW to uncommitted 
COOKED!  This is just smoke and mirrors.

Art S. Kagel

> Best Regards
> 
<SNIP>
0
Art
5/12/2005 4:42:58 PM
Reply:

Similar Artilces:

Re: raw vs. cooked files under linux
SCO -> Linux ... good ;) If you want to upgrade to IDS 10.0 then better use Suse 9 'cause IDS use Kernel 2.6 specific features like KAIO. There is a nice document on IDS 10 and Linux on IBM site (can't remember link...). Regarding cooked vs. raw devices O bet for raw devices, they are very easy to configure in linux and give you some benefits like KAIO for example. J. Heinz Weitkamp escribi�: > Hi all, > > i am an linux newbie. > I must shift our Databases from SCO Unix (IDS 7.31 UD4) to SuSE Linux > Enterprise Edition 8.1 (2.4.21-190-smp glibc 2.2.5) or (8.2) or 9 and SuSE > Linux Professional 9.3 (2.6.11.4 glibc 2.3.4) (IDS 7.31 UD8). > > Later on i want to upgrade (in place) to IDS 10.0. > Which Version of SuSE Linux Enterprise Edition must I use (8.? , 9 ..)? > > Under Sco Unix we use raw-devices. > Is it better to use cooked files or raw files under these Linux-Versions? > A colleague mentioned, it can be, that later releases of Linux don't support > raw-devices. Is that correct? > > Experiences or comments are welcome. > Thanks in advance. > (Excuse my poor school-english) > > Heinz > > sending to informix-list > sending to informix-list ...

RE: raw vs. cooked files under linux #6
When you compare cooked versus raw here, are you focusing on Linux only, or talking about this in general - other Unix platforms as well ? -----Original Message----- From: owner-informix-list@iiug.org [mailto:owner-informix-list@iiug.org] On Behalf Of Data Goob Sent: 11 May 2005 03:08 PM To: informix-list@iiug.org Subject: Re: raw vs. cooked files under linux If you are in a slow-change environment raw-files are not a bad thing. They create extra problems on the management side that cooked files do not. Fast-changing environments create more demand on your time so why put yourself into extra work with raw files? They don't give you enough of a difference in performance to really warrant them unless you are really demanding that extra 10%. Cooked files are also also more flexible on backups, you can back the dbspaces up with a regular backup tool or use the Informix backup tools, but raw files will have to be backed up with Informix tools only. SLES9 is quite a sea change from SLES8, and you could probably get away without SLES and go with the workstation Pro 9.3 if you are on a budget. SLES has better support for SCSI than the workstation versions and you must buy support or the online update doesn't work. It's an enterprise lock-in you might not need so try before you buy. SLES9 can be downloaded for trial, so give it a spin. You should be able to download IDS 10 for trial as well, thus no out of pocket to try it before pay...

RE: raw vs. cooked files under linux #3
We're about to setup a new client on IDS 10 on Linux and I've now seen two recommendations for SUSE in relation to IDS 10. Does anyone have experience with Redhat Enterprise Edition and IDS 10? Any issues there that we should be aware of? Bill Weaver Director of Engineering Amicus, Inc. 512-531-3463 (office) 512-531-3401 (fax) -----Original Message----- From: owner-informix-list@iiug.org [mailto:owner-informix-list@iiug.org] On Behalf Of Sandor Szabo Sent: Wednesday, May 11, 2005 6:31 AM To: Heinz Weitkamp Cc: user-group Informix (E-Mail); owner-informix-list@iiug.org Subject: Re: raw vs. cooked files under linux Dear Heinz, I would suggest that you pick SUSE SLES 9 This SUSE version works perfectly with IDS 10. Please check the following link: http://www-306.ibm.com/software/data/informix/linux/ids.html With IDS 10 you have the freedom of choice to use cooked or raw devices with KAIO This link should answer your raw device support question http://www-128.ibm.com/developerworks/db2/library/techarticle/dm-0503szabo/? ca=dgr-lnxw41IDSTen Let me know if you have any additional questions. bye Sandor IBM Informix Development Munich Information Management Hollerithstra�e 1 81829 Munich Germany Phone: +49 89 4504 1429 "Heinz Weitkamp" <heinz.wei...

Re: raw vs. cooked files under linux #7
We are running 24 instances of IDS 9.4 on 24 machines - about 95% raw chunks - no problems so far -Can you explain some of your reasoning below? What specifically are the risks and additional management? ----- Original Message ----- From: "Data Goob" <datagoob@netscape.net> To: <informix-list@iiug.org> Sent: Wednesday, May 11, 2005 9:08 PM Subject: Re: raw vs. cooked files under linux > Art S. Kagel wrote: > > Heinz Weitkamp wrote: > > > >> Hi all, > > > > > > Go RAW all the way. Contrary to the Goob's understanding my own testing > > shows RAW devices to be 15-25% faster than COOKED devices which are in > > turn 10-15% faster than filesystem files, even on the best filesystem > > implementations that adds up to a 25% improvement. Well worth any minor > > inconveniences in my book. > > OK so you get some speed increases, but also the additional management > of linking raw files. The original post indicated he was a newbie. > Raw files are not necessarily difficult, but they ARE optional and not > something I would entrust to a newbie. It's a great learning experience, > but who's paying for it?? > > But another point to make is that in many SAN environments, raw files are > a complete waste of time, with the data already being spread out over many > drives, and, with speeds typically in the 10-15K rpm ran...

RE: raw vs. cooked files under linux #9
-----Original Message----- From: owner-informix-list@iiug.org [mailto:owner-informix-list@iiug.org] On Behalf Of John Carlson Sent: 13 May 2005 03:48 AM > >Almost zero? Will you be in Denver? Come see how many Informix DBAs attend >my own sessions or Jonathan's or Lester's. There are a lot more of us out >there then you imagine. > [snip] >As for the job security comments . . . just remember that there are >still a goodnumber of DBAs who will not be able to go to Denver. True, my company will only allow me to go places I can drive to (short distance) - very sad ........ sending to informix-list ...

Re: raw vs. cooked files under linux #2
Dear Heinz, I would suggest that you pick SUSE SLES 9 This SUSE version works perfectly with IDS 10. Please check the following link: http://www-306.ibm.com/software/data/informix/linux/ids.html With IDS 10 you have the freedom of choice to use cooked or raw devices with KAIO This link should answer your raw device support question http://www-128.ibm.com/developerworks/db2/library/techarticle/dm-0503szabo/?ca=dgr-lnxw41IDSTen Let me know if you have any additional questions. bye Sandor IBM Informix Development Munich Information Management Hollerithstra�e 1 81829 Munich Germany Phone: +49 89 4504 1429 "Heinz Weitkamp" <heinz.weitkamp@w estfleisch.de> To Sent by: "user-group Informix \(E-Mail\)" owner-informix-li <informix-list@iiug.org> st@iiug.org cc Subject 11.05.2005 10:48 raw vs. cooked files under linux ...

Re: raw vs. cooked files under linux #10
DA Morgan said: > Data Goob wrote: >> Art, >> >> Your points are well made, and I'm willing to admit I could be >> misinformed, but quite honestly you are working in an exceptional >> environment compared to a lot of environments out there. You >> make the assertion that everyone should simply get it when it >> comes to disk drives, and quite honestly very few of us can keep >> up with this. You also make the assumption that it is easy when >> you should see what I have to work with when using sys admins--you >> even allude to the jr DBA who got it wrong. Imagine that this is >> 95% of the environments out there. And as far as misinforming >> people bang on EMC, they even preach their RAID5 is 'better', >> so we even have to deal with that as well as a lot of other >> vendor misinformation. > > There actually is one implementation of RAID5 that stands up to testing > as far superior to others and that is Apple Computer's XServe RAID. > These units have two XOR engines and come remarkably close in > performance to 0+1 (within a few percentage points) and also have the > advantage of having minimal loss of performance when running in degraded > mode. So far this year most of the storage I have worked with has been > Apple's and it is a poorly kept secret that Oracle Corp. is now using > them in their data center in...

Re: raw vs. cooked files under linux #8
How does one use O_SYNC with and ext3 filesystem on Linux? I don't see it mentioned anywhere. Normally I use raw but SA and management would rather use cooked. They would also rather use Postgres but that another story. Thanks >You assert that cooked files are as fast today as RAW on SANs. Have YOU >tested it? Are you comparing RAW to COOKED with O_SYNC? Or are you >depending on the vendor stats? Not me, not with my reputation, user >satisfaction, and my job at stake. > > sending to informix-list Peter J Diaz de Leon wrote: > How does one use O_SYNC with and ext3 filesystem on Linux? I don't see > it mentioned anywhere. Normally I use raw but SA and management > would rather use cooked. They would also rather use Postgres but that > another story. > Probably the same way one does for any filesystem, by adding the O_SUNC flag to the oflag argument to the open() system call. So: out_handle = open( "/myfs/mychunkfile", O_WRONLY | O_SYNC ); But isn't ext3 a journaled filesystem? One would first off, not want to use a journaled filesystem for IDS chunks for several reasons: 1- Performance, the overhead of writing the journal records 2- No gain for the pain. You can NEVER use the journal to reproduce the file because it's state would always be to the moment. If the disk crashes you'll have to restore from the IDS archives anyway. Art S. Kag...

RE: raw vs. cooked files under linux #5
We use raw chunks on all our unix versions. Most are Linux. We find it easier to set up and manage raw devices, including under Linux. We also have a SAN environment running RAW devices under Linux - no problems at all. We have not had any issues and that includes recovering systems. Its all very simple when using RAW devices. MW > -----Original Message----- > From: owner-informix-list@iiug.org > [mailto:owner-informix-list@iiug.org] On Behalf Of Data Goob > Sent: Thursday, 12 May 2005 1:09 p.m. > To: informix-list@iiug.org > Subject: Re: raw vs. cooked files under linux > > Art S. Kagel wrote: > > Heinz Weitkamp wrote: > > > >> Hi all, > > > > > > Go RAW all the way. Contrary to the Goob's understanding my own > > testing shows RAW devices to be 15-25% faster than COOKED devices > > which are in turn 10-15% faster than filesystem files, even on the > > best filesystem implementations that adds up to a 25% improvement. > > Well worth any minor inconveniences in my book. > > OK so you get some speed increases, but also the additional management > of linking raw files. The original post indicated he was a newbie. > Raw files are not necessarily difficult, but they ARE optional and not > something I would entrust to a newbie. It's a great learning > experience, but who's paying for it?? > > But anot...

Re: Raw vs. Cooked files
Yes even with DB2 UDB you can get 20-25% improvement going with RAW over Cooked. If you use cooked the IO has to be written by the Unix Operating system - another set of commands to execute. Raw is controlled by the database. Why o Why Are we still having this discussion? Rob Vorbroker --- Rajesh Kapur <rkapur@mpr.org> wrote: > Hello, > > I am installing IDS 10 on RHEL3 on an intel x86 > server. I need to make the > choice between raw devices and cooked files. > I found the following information in the FAQ's. Is > this information still > accurate? Do cooked files continue to be > significantly slower than the raw > files? I have heard that UNIX file systems have > improved over the years and > the performance divide between raw and cooked files > has narrowed > considerabley. Is it true? > > Thanks! > - Rajesh > -------------------------------------------------------------------------------------- > > 6.38 Is raw disk faster than cooked files? > On 22nd Jun 1998 kagel@bloomberg.net (Art S. Kagel) > wrote:- > > ....................the safety issue of cooked files > is no longer a problem. > The big problem with cooked files still is > performance. All writes and reads > to/from cooked files MUST go through the UNIX buffer > cache. This means an > additional copy from the server's output buffer to > the UNIX buffe...

raw vs. cooked files under linux
Hi all, i am an linux newbie. I must shift our Databases from SCO Unix (IDS 7.31 UD4) to SuSE Linux Enterprise Edition 8.1 (2.4.21-190-smp glibc 2.2.5) or (8.2) or 9 and SuSE Linux Professional 9.3 (2.6.11.4 glibc 2.3.4) (IDS 7.31 UD8). Later on i want to upgrade (in place) to IDS 10.0. Which Version of SuSE Linux Enterprise Edition must I use (8.? , 9 ..)? Under Sco Unix we use raw-devices. Is it better to use cooked files or raw files under these Linux-Versions? A colleague mentioned, it can be, that later releases of Linux don't support raw-devices. Is that correct? Experiences or comments are welcome. Thanks in advance. (Excuse my poor school-english) Heinz sending to informix-list If you are in a slow-change environment raw-files are not a bad thing. They create extra problems on the management side that cooked files do not. Fast-changing environments create more demand on your time so why put yourself into extra work with raw files? They don't give you enough of a difference in performance to really warrant them unless you are really demanding that extra 10%. Cooked files are also also more flexible on backups, you can back the dbspaces up with a regular backup tool or use the Informix backup tools, but raw files will have to be backed up with Informix tools only. SLES9 is quite a sea change from SLES8, and you could probably get away without SLES and go with the workstation Pro 9.3 if you are on a budget. SLES has...

IDS 10 on RHEL4 Linux: I/O benchmark of RAW DEVICES vs BLOCK DEVICES vs COOKED FILES
Hello there, this is a long message (sorry in advance) which contains some hopefully interesting information. There are also some questions at the end. I have this nice new Red Hat Enterprise Linux 4 machine, it's an HP DL 385 with 8 internal 10K RPM SAS disks, 2 dual-core AMD Opteron CPUs and 4 GB RAM. Machine features 256 Mb of cache on the disks controller. Here it is: http://tinyurl.com/p6k6y I have installed Informix Dynamic Server 10.00.FC4 (64 bit). I bumped into this IBM article: http://tinyurl.com/gfovb, where it says that Informix IDS 10 on RHEL4 Linux is able to access block devices in unbuffered fashion -like it does for the raw devices- using KAIO (Kernel Asynchronous I/O). The server is brand new, and I can play with it a bit before it goes into production, so I decided to do some performance testing of RAW vs BLOCK vs COOKED, to see which one was better from the I/O performance point of view. Some definitions, first: * RAW DEVICE or CHARACTER DEVICE = devices through which data is transmitted one character at a time, using unbuffered input and output routines; each character is read from, or written to, the device immediately; such devices are disk partitions without a file system on them, and are therefore managed from outside the operating system using direct access and bypassing the OS layer and its cache * BLOCK DEVICE = devices through which data is transmitted in the form of blocks, the most sign...

Re: Raw vs. Cooked files #3
HOWEVER, Informix IDS does not user the OS buffering. It opens all COOKED chunks with O_SYNC which bypasses most of the benefits of the buffering in favor of added data integrity. Art S. Kagel ----- Original Message ----- From: Martin Fuerderer <MARTINFU@de.ibm.com> At: 5/18 4:27 Hi, I'd say the correct answer to the question is: "It depends." -------------------------------------------------------------------- The implementation of file system I/O buffering by the OS can be very efficient on Linux. But for that the OS needs sufficient memory for the buffering. And this is what makes the "It depends". Therefore there are 2 extremes: a) almost all physical memory is used by IDS in the form of shared memory (allocated SHM segments listed by "onstat -g seg"). A good indication for this scenario is a machine that uses little swap space, because it already has swapped out some minor "applications". The system is at the (sensible) limit of its memory usage. In this scenario, raw devices and the utilization of KAIO (available with IDS 10.00.UC1 on Linux) will in general perform better than cooked files. b) there's a lot of unused physical memory left over, even though IDS is configured optimally (e.g. BUFFERS is big enough, etc.). A recent example of such a scenario was 32-bit IDS installed on a 64-bit Linux, where IDS can use only ~4 GB of the ava...

Re: Measurements: (was: Raw vs. Cooked files)
Hi, maybe. :) This work was done by zSeries people (which are in a different department/organization from us in IBM Informix Data Management ...). They used IDS mainly for their purpose (to show zLinux capabilities like scaling with a real, commercially available application). Therefore we cannot directly demand from them to do this again with IDS 10.00. However, there is a possibility, that for their next round of measurements they will use IDS 10.00 (as they made good experiences with IDS 9.4). Though in that case I would not expect results before sometime next year .... Regards, Martin -- Martin Fuerderer IBM Informix Development Munich, Germany Information Management owner-informix-list@iiug.org wrote on 29.06.2005 02:31:50: > > Can we get this done with IDS 10? > > Martin Fuerderer wrote: > > Hi, > > > > if you are interested : > > > > http://www.ibm.com/servers/eserver/zseries/library/techpapers/pdf/gm130667.pdf > > > > (There's a part where they tested raw devices vs. file system files as > > well. > > These measurements were all done on zSeries, not by myself as I did not > > get to do such extensive work. But this document is really extensive ....) > > > > Regards, > > Martin > > -- > > Martin Fuerderer > > IBM Informix Development Munich, Germany > > Information Management sending to i...

AW: raw vs. cooked files under linux
Thank you very much, Jean and Sandor, for your quick answers. One question more: I saw in the List on link: http://www-306.ibm.com/software/data/informix/linux/ids.html that IDS 7.31.UD8 don't support SUSE SLES9. Do you know, whether IDS 7.31.UD8 works under SUSE SLES9, because first i want to install IDS 7.31.UD8 on Linux and later IDS 10.0 without (if possible) changing the Linux Version? Thanks. Heinz -----Urspr�ngliche Nachricht----- Von: Sandor Szabo [mailto:sandor.szabo@de.ibm.com] Gesendet: Mittwoch, 11. Mai 2005 13:31 An: Heinz Weitkamp Cc: user-group Informix (E-Mail); owner-informix-list@iiug.org Betreff: Re: raw vs. cooked files under linux Dear Heinz, I would suggest that you pick SUSE SLES 9 This SUSE version works perfectly with IDS 10. Please check the following link: http://www-306.ibm.com/software/data/informix/linux/ids.html With IDS 10 you have the freedom of choice to use cooked or raw devices with KAIO This link should answer your raw device support question http://www-128.ibm.com/developerworks/db2/library/techarticle/dm-0503szabo/? ca=dgr-lnxw41IDSTen Let me know if you have any additional questions. bye Sandor IBM Informix Development Munich Information Management Hollerithstra�e 1 81829 Munich Germany Phone: +49 89 4504 1429 "Heinz Weitkamp" <heinz.weitkamp@w estfleisch.de> ...

Re: Raw vs. Cooked files #2
Hi, I'd say the correct answer to the question is: "It depends." -------------------------------------------------------------------- The implementation of file system I/O buffering by the OS can be very efficient on Linux. But for that the OS needs sufficient memory for the buffering. And this is what makes the "It depends". Therefore there are 2 extremes: a) almost all physical memory is used by IDS in the form of shared memory (allocated SHM segments listed by "onstat -g seg"). A good indication for this scenario is a machine that uses little swap space, because it already has swapped out some minor "applications". The system is at the (sensible) limit of its memory usage. In this scenario, raw devices and the utilization of KAIO (available with IDS 10.00.UC1 on Linux) will in general perform better than cooked files. b) there's a lot of unused physical memory left over, even though IDS is configured optimally (e.g. BUFFERS is big enough, etc.). A recent example of such a scenario was 32-bit IDS installed on a 64-bit Linux, where IDS can use only ~4 GB of the available 24 GB of physical memory. In such a scenario, the OS can allocate a lot of memory for file system I/O buffering and this will make file system I/O (i.e. for chunks on cooked files) very efficient. In this scenario it will normally perform better than raw devices and KAIO. This i...

Re: Measurements: (was: Raw vs. Cooked files) #2
Hi, > Why is NOAGE set to 0? Surely it should be set to 1? For this work the machine was dedicated solely to IDS server. No other applcations were running on the machine. Therefore NOAGE 1 vs. 0 would not have made much of a difference as there were no other processes to compete with IDS for CPU-time. Regards, Martin -- Martin Fuerderer IBM Informix Development Munich, Germany Information Management owner-informix-list@iiug.org wrote on 29.06.2005 02:32:39: > > Nice but.... > > Can we get this done with IDS 10? > > Set RESIDENT to -1 to make everything resident. > > Why is NOAGE set to 0? Surely it should be set to 1? > > > Martin Fuerderer wrote: > > Hi, > > > > if you are interested : > > > > http://www.ibm.com/servers/eserver/zseries/library/techpapers/pdf/gm130667.pdf > > > > (There's a part where they tested raw devices vs. file system files as > > well. > > These measurements were all done on zSeries, not by myself as I did not > > get to do such extensive work. But this document is really extensive ....) > > > > Regards, > > Martin > > -- > > Martin Fuerderer > > IBM Informix Development Munich, Germany > > Information Management > > > sending to informix-list Martin Fuerderer schrieb: > Hi, > > >>Why is NOAGE set to 0? Surely i...

Re: Relocate Informix database from Raw slices to filesystem #4
True there are issues but it's a easy solution to the problem, gets round the 'can not rename chunks' issue, no need to hassle about mirroring etc But the original post said going to internal disks so there will not be a name clash. On 10/17/05, Paul Watson <pwatson@irace.com> wrote: > > On 17 Oct 2005 15:26:36 -0700, david@smooth1.co.uk <david@smooth1.co.uk> > wrote: > > > > > > Paul Watson wrote: > > > If the device is 'not going to exist anymore' why not dd the data from > > the > > > old slice to the internal disk and then > > > just link the 'not going to exist anymore device' to the new location > > > > > > > > > > > > I'm guess this a Solaris platform..and then someone runs devfsadm that > > rebuilds /dev and removes your links! > > > > Or someone adds a new device and Solaris wants to use that name! > > > > Bad idea to store stuff under /dev. That directory is for the OS > > to control not you. > > > > If you are running on IDS 9.40 or IDS 10 then ontape has the ability to > > rename chunks during a restore. > > > > http://www-128.ibm.com/developerworks/db2/library/techarticle/dm-0405fan/ > > > > > > > > > -- > Paul Watson > Tel: +44 7818 003457 > Fax: +44 1436 678693 > ...

RE: Raw device VS cooked file
In my experience with AIX. The new enhanced journal file system (JFS2), there are file system option to use SIO or DIO. Using DIO (direct io) gives pretty close results to raw partition with Oracle and Informix. I am not sure if Solaris, HPUX or Linux has any option similar to AIX Basim Chafik Senior Systems Analyst IBM Certified Advanced Technical Expert (CATE) 1.800.688.4895 basim.chafik@plx.com plexus (Division of BancTec) -----Original Message----- From: Rupan3rd [mailto:rupan.nospam.3rd@hotmail.com] Sent: Thursday, January 12, 2006 11:18 AM To: informix-list@iiug.org Subject: Re: Raw device VS cooked file - any benchmark recently done? I know what pros and cons of using raw might be. My need is to have the best performance for my OLTP system, therefore I was wordering if and in what cases it might be actually true that there are disk devices able to achieve "I/O caching for cooked writes" that "can provide similar if not better performance" than raw areas. Still the documentation does not explicitly say if the cooked READS would be that good as well nor how to benchmark nor the effect of the fragmentation. Regards Rupan3rd malcolm weallans wrote: > I would really like to see any evidence that people have on this issue. I > have a number of systems that are running with acceptable performance on raw > disks. But could we be doing better and at what cost? > > But from almost ...

Re: hi Cant load informix online 5.2 on linux -8 ( rpm file) #4
yeh all symbolic lins have fallen completely, Now what to do regards RATHORE ----- Original Message ----- From: "Ronald Cole" <ronald@forte-intl.com> To: <informix-list@iiug.org> Sent: Wednesday, December 31, 2003 12:01 PM Subject: Re: hi Cant load informix online 5.2 on linux -8 ( rpm file) > "Neil Truby" <neil.truby@ardenta.com> writes: > > Install it in /opt/informix as required to do by the bug, then use find > > . -print | cpio -pvd or some other method to copy the contents of > > /opt/informix to /usr/inf...

Re: Database Lock vs Database Freeze #4
And just to add to the confusion, what Nancy said about database freeze=20 vs. database lock has the meaning reversed in my company. Maybe it's=20 because we use Clintrial instead of Oracle Clinical. Bob Abelson HGSI 240 314 4400 x1374 bob=5Fabelson@hgsi.com "Nancy Brucken" <brucken@PROVIDE.NET>=20 Sent by: "SAS(r) Discussion" <SAS-L@LISTSERV.UGA.EDU> 04/24/2007 02:05 AM Please respond to "Nancy Brucken" <brucken@PROVIDE.NET> To SAS-L@LISTSERV.UGA.EDU cc Subject Re: Database Lock vs Database Freeze Hi Madan, To build on what Jim...

Raw vs. Cooked files
Hello, I am installing IDS 10 on RHEL3 on an intel x86 server. I need to make the choice between raw devices and cooked files. I found the following information in the FAQ's. Is this information still accurate? Do cooked files continue to be significantly slower than the raw files? I have heard that UNIX file systems have improved over the years and the performance divide between raw and cooked files has narrowed considerabley. Is it true? Thanks! - Rajesh -------------------------------------------------------------------------------------- 6.38 Is raw disk faster than cooked files? On 22nd Jun 1998 kagel@bloomberg.net (Art S. Kagel) wrote:- .....................the safety issue of cooked files is no longer a problem. The big problem with cooked files still is performance. All writes and reads to/from cooked files MUST go through the UNIX buffer cache. This means an additional copy from the server's output buffer to the UNIX buffer page then a synchronous write to disk. This is opposed to a write to a way file where the server's output buffer is written directly to disk without the intervening copy. This is just faster. Anyone who has written anything that can test this can attest to the difference in speed. Here are my test results: FileType Sync? Times (real/user/system) 2run avg --------------- ----- ----------------------------------------- Filesystem file N 14.40/3.70/2.52 Y 15.02/3.61/2.63 Cooked ...

Re: BIND 9.4.x vs 9.6.x
Looking at the publically available parts of SunSolve there are at least bug reports about it. Requires Support Contract tmp_mkdir()/xmemfs_mkdir() inconsistent with other xxxfs_mkdir() functions. | Open in a new window bug 6253984 http://sunsolve.sun.com/search/document.do?assetkey=1-1-6253984-1 - Sep 10, 2007 Requires Support Contract tmp_mkdir()/xmemfs_mkdir() inconsistent with other xxxfs_mkdir() functions. | Open in a new window bug 2152581 http://sunsolve.sun.com/search/document.do?assetkey=1-1-2152581-1 - Sep 10, 2007 I don't have a copy of the POSIX standard tha...

Re: TURBOIMAGE error- Directed end of file
Michael, ****** Your link is to Version 6, dated August, 1997, and I have a copy of Versi= on 8,=20 dated July, 2001. Both say the same, but in different places. 'Directed end of file' is Exceptional Condition #13. Appendix A, page A-62 (physical page 548 in Version 6) explains: AND Appendix A, page 585 (Version 8) explains: 13 MESSAGE Directed end of file MEANING DBGET has been called for a directed read with a record number greater than the capacity of a dataset. ACTION Appropriate action depends in program design. So t...

Web resources about - Re: raw vs. cooked files under linux #4 - comp.databases.informix

Punky Brüster – Cooked on Phonics - Wikipedia, the free encyclopedia
Punky Brüster – Cooked on Phonics (a pun of the literacy program Hooked on Phonics ) is the debut studio album by Canadian musician Devin Townsend ...

The cooked veggieducken - Flickr - Photo Sharing!
Explore The Sporkful's photos on Flickr. The Sporkful has uploaded 46 photos to Flickr.

Chef dismembered, cooked girlfriend in Brisbane apartment
A young chef had been boiling his girlfriend's dismembered body parts in his Brisbane apartment when he was discovered by police on Saturday ...


Carbon tax not all it was cooked up to be
Carbon tax not all it was cooked up to be

HomeCooked app: Buy your neighbour’s home-cooked food instead of takeaways - The Courier-Mail Search ...
WE COULD soon be bidding farewell to the fish and chip shop and saying ta-ta to the takeaway Thai if a plan to transform the way we eat, in the ...


Tanzanian man 'beheaded and cooked'
A Tanzanian man has been beheaded and cooked following accusations of adultery.

They came, they cooked, they conquered: it's all about being Italian
They came, they cooked, they conquered: it's all about being Italian

The books are being cooked on 457 visas
The foreign worker program is being rorted just as the student visas were.

Resources last updated: 3/1/2016 5:39:08 AM