Hi all, I have a problem that is really puzzling me and annoying my 
customers, and I am not sure even on what to investigate first. Any hint 
will be appreciated.
Scenario: a networked environment in which an application compiled with 
xharbour 0.99 + wvw manages some databases with about 50.000/60.000 
records from XP clients via a Samba share on a Linux server. Every month 
about 1500/2000 new records are added. New records are marked by a flag 
as 'news' and are marked as 'old' at the end of their monthly lifetime. 
A conditional index allows for a browse to  edit only 'new' records. 
Other indexes are not conditional and allow for a full search and edit. 
The RDD is DBFCDX.
Recently a server-side CGI running under Linux (Slackware, kernel 2.6) 
has been added that allows for external users to interact via a web 
interface from outside the intranet and update the same archives.
I have no problems at all from the point of view of data management, 
shares, concurrent access, and everything works as expected, but the 
opening of the files has become very slow for the users in the intranet, 
to the point I must admit that the whole thing is too slow for a 
professional environment. The app waits some 10-15 seconds to open the 
archives, and this lag is really annoying. Every single operation opens 
and then closes the databases, and the app has been working quickly and 
  smoothly for years. Now when an user chooses an operation from the 
menu there is this incredible lag, and I have not been able to figure 
out what's happening.
First of all I tested the speed of the network, figuring a network 
problem (switch, cable...) but with netspeed and isolating the PC's on a 
single 'piece' of network I was not able to detect any problem. (I 
launched some file transfer from/to clients to/from the server, with 
netspeed in background measuring the speed.)
Then I put off the router, blaming the external accesses (maybe a 
different lock scheme from the CGI, running under Linux...)
Again, the behavior did not change. The speed of data transfer in file 
opening is a fraction of the regular file transfer speed according to 
netspeed. What does it mean?
Then I asked to reduce the overall number of records, forcing the 
'administrator' (I am the real administrator, ehm) to store the 'dead' 
records to the 'history' archives (those are archives where the records 
are amassed at the end of their lifecycle, and are usually not opened at 
all by the regular users. Only the 'administrator' can access them via a 
reserved function, another browse in fact, so they are irrelevant).
At present time, nothing seems to impact the slow performance. I wonder 
if it can be a Samba problem, even if I was not able to detect it. Next 
try will be a  'mirror' test program with deflated databases in another 
directory. Can the size of a set of DBF influence the time needed to 
access them at opening time? I never heard about it in Clipper, nor in 
xHarbour.
I hope to have (pant!) explained clearly the problem. I am considering 
to upgrade to xHarbour 1.21.xxxx but I have to be quite sure that wvw 
works as expected under Windows, and I'll have to upgrade the Linux 
development environment too... so BEFORE to go for that I'll be very 
happy to hear some advice!
TIA
D. Campagna

Dear D.Campagna:

On Jan 13, 3:18=A0pm, "D.Campagna" <ynnadre...@tiscalinet.it> wrote:
> Hi all, I have a problem that is really puzzling me and
> annoying my customers, and I am not sure even on
> what to investigate first. Any hint will be appreciated.
> Scenario: a networked environment in which an
> application compiled with xharbour 0.99 + wvw
> manages some databases with about 50.000/60.000
> records from XP clients via a Samba share on a
> Linux server. Every month about 1500/2000 new
> records are added. New records are marked by a flag
> as 'news' and are marked as 'old' at the end of their
> monthly lifetime.  A conditional index allows for a
> browse to =A0edit only 'new' records.  Other indexes are
> not conditional and allow for a full search and edit.
> The RDD is DBFCDX.

Move to SQL.

> Recently a server-side CGI running under Linux
> (Slackware, kernel 2.6) has been added that allows
> for external users to interact via a web interface
> from outside the intranet and update the same
> archives.

Make the internet version a clone of the intranet's files, and update
the clone copies overnight.  Internet clients are used to waits.

> development environment too... so BEFORE to
> go for that I'll be very happy to hear some advice!

This is my nickel's worth of free advice.

David A. Smith

"D.Campagna" <ynnadrebyc@tiscalinet.it> ha scritto nel messaggio 
news:hilgrc$m4b$1@nnrp-beta.newsland.it...

> but the opening of the files has become very slow for the users
> in the intranet,

I noticed the same problem. I compared the same app compiled in March 2008 
and today and found the former much faster in opening DBFs.

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

dlzc ha scritto:
> Dear D.Campagna:
> 
> On Jan 13, 3:18 pm, "D.Campagna" <ynnadre...@tiscalinet.it> wrote:
>> Hi all, I have a problem that is really puzzling me and
>> annoying my customers, and I am not sure even on
>> what to investigate first. Any hint will be appreciated.
>> Scenario: a networked environment in which an
>> application compiled with xharbour 0.99 + wvw
>> manages some databases with about 50.000/60.000
>> records from XP clients via a Samba share on a
>> Linux server. Every month about 1500/2000 new
>> records are added. New records are marked by a flag
>> as 'news' and are marked as 'old' at the end of their
>> monthly lifetime.  A conditional index allows for a
>> browse to  edit only 'new' records.  Other indexes are
>> not conditional and allow for a full search and edit.
>> The RDD is DBFCDX.
> 
> Move to SQL.

(Ugh) This is not going to be a quick solution, I am new to SQL...
> 
>> Recently a server-side CGI running under Linux
>> (Slackware, kernel 2.6) has been added that allows
>> for external users to interact via a web interface
>> from outside the intranet and update the same
>> archives.
> 
> Make the internet version a clone of the intranet's files, and update
> the clone copies overnight.  Internet clients are used to waits.

I forgot to say that the Linux CGI works as expected, that is to say 
it's very fast. No lags here.

> This is my nickel's worth of free advice.
> 
> David A. Smith

Thanks, David. I suspected it's time for me to begin to practice with SQL...
Dan

Enrico Maria Giordano ha scritto:
> "D.Campagna" <ynnadrebyc@tiscalinet.it> ha scritto nel messaggio 
> news:hilgrc$m4b$1@nnrp-beta.newsland.it...
> 
>> but the opening of the files has become very slow for the users
>> in the intranet,
> 
> I noticed the same problem. I compared the same app compiled in March 2008 
> and today and found the former much faster in opening DBFs.
> 
> EMG

But, let me understand: you have recompiled with a new xHarbour version?
I haven't. The program running on XP and Windows 2000 clients is still 
the same. And, as told to David, the Linux CGI is a flash.

And, by the way, how have you managed / planned to solve the problem?
Dan

"D.Campagna" <ynnadrebyc@tiscalinet.it> ha scritto nel messaggio 
news:himkf9$97m$1@nnrp-beta.newsland.it...

> But, let me understand: you have recompiled with a new xHarbour version?

Yes. I always use latest xHarbour from CVS.

> I haven't. The program running on XP and Windows 2000 clients is still the 
> same. And, as told to David, the Linux CGI is a flash.

So yours is a different problem, I suppose.

> And, by the way, how have you managed / planned to solve the problem?

No, I only alleviated it by reducing unnecessary USE/CLOSE.

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

Dear D.Campagna:

On Jan 14, 1:19=A0am, "D.Campagna" <ynnadre...@tiscalinet.it> wrote:
> dlzc ha scritto:
> > On Jan 13, 3:18 pm, "D.Campagna" <ynnadre...@tiscalinet.it> wrote:
> >> Hi all, I have a problem that is really puzzling me and
> >> annoying my customers, and I am not sure even on
> >> what to investigate first. Any hint will be appreciated.
> >> Scenario: a networked environment in which an
> >> application compiled with xharbour 0.99 + wvw
> >> manages some databases with about 50.000/60.000
> >> records from XP clients via a Samba share on a
> >> Linux server. Every month about 1500/2000 new
> >> records are added. New records are marked by a flag
> >> as 'news' and are marked as 'old' at the end of their
> >> monthly lifetime. =A0A conditional index allows for a
> >> browse to =A0edit only 'new' records. =A0Other indexes are
> >> not conditional and allow for a full search and edit.
> >> The RDD is DBFCDX.
>
> > Move to SQL.
>
> (Ugh) This is not going to be a quick solution, I am
> new to SQL...

Same here.  It is only the recent past technology, where DBFCDX is
distant past technology.

> >> Recently a server-side CGI running under Linux
> >> (Slackware, kernel 2.6) has been added that allows
> >> for external users to interact via a web interface
> >> from outside the intranet and update the same
> >> archives.
>
> > Make the internet version a clone of the intranet's
> > files, and update the clone copies overnight.
> >=A0Internet clients are used to waits.
>
> I forgot to say that the Linux CGI works as
> expected, that is to say it's very fast. No lags here.

It was implied and I understood that.  I suspect the "CGI overhead"
may be causing the slow down in other operations.  So if there is no
overhead associated with locally accessed files...

> > This is my nickel's worth of free advice.
>
> Thanks, David. I suspected it's time for me to
> begin to practice with SQL...

You and me both...

David A. Smith

"dlzc" <dlzc1@cox.net> ha scritto nel messaggio 
news:6474dcc7-d027-4bbd-b7c8-bce9069df5c2@s31g2000yqs.googlegroups.com...

> Same here.  It is only the recent past technology, where DBFCDX is
> distant past technology.

SQL is much older than DBFCDX.

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

On Jan 14, 10:30=A0am, "Enrico Maria Giordano"
<e.m.giord...@emagsoftware.it> wrote:
> "dlzc" <dl...@cox.net> ha scritto nel messaggionews:6474dcc7-d027-4bbd-b7=
c8-bce9069df5c2@s31g2000yqs.googlegroups.com...
>
> > Same here. =A0It is only the recent past technology,
> > where DBFCDX is distant past technology.
>
> SQL is much older than DBFCDX.

You are correct.  It even predates dBase II (so even early DBFNTX) by
a few years.

But you will agree it is a better solution for "mixed access"
architecture, I think.

David A. Smith

David & Enrico said:

>> Same here.  It is only the recent past technology, where DBFCDX is
>> distant past technology.
>
> SQL is much older than DBFCDX.
>
> EMG

David:

    I'm older than *any* technology around today (except maybe the 
automobile) :((

    I 'sampled' SQL many years ago, and found it daunting and 'remote'. 
That is,  a programmer couldn't get really *close* to the actual data inside 
the SQL database -- it seems to have a wall around the actual files).  With 
DBFCDX and its predecessors (DBFNTX, etc),  I feel much closer to the data 
than with the (frightening) SQL database protocol.

    Is modern SQL faster than CDX stuff ??

-Mel Smith 

"dlzc" <dlzc1@cox.net> ha scritto nel messaggio 
news:67d03f0f-caf3-479d-9aaf-15f41a1b8840@k19g2000yqc.googlegroups.com...

> But you will agree it is a better solution for "mixed access"
> architecture, I think.

Yes... and no. I think SQL and client-server are obsolete technologies 
borned to overcome networks slowness. They are losting their purpose day 
after day.

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

"Mel Smith" <med_cutout_syntel@aol.com> ha scritto nel messaggio 
news:7r939pFh4nU1@mid.individual.net...

>    I'm older than *any* technology around today (except maybe the 
> automobile) :((

I'm class 1960, and you?

>    I 'sampled' SQL many years ago, and found it daunting and 'remote'. 
> That is,  a programmer couldn't get really *close* to the actual data 
> inside the SQL database -- it seems to have a wall around the actual 
> files).  With DBFCDX and its predecessors (DBFNTX, etc),  I feel much 
> closer to the data than with the (frightening) SQL database protocol.

Yes, but that is considered an advantage from the security point of view.

>    Is modern SQL faster than CDX stuff ??

Yes and no. It depends. It's not so easy to compare those two deeply 
phylosophycally different technologies.

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

Mel Smith ha scritto:
> David & Enrico said:
> 
>>> Same here.  It is only the recent past technology, where DBFCDX is
>>> distant past technology.
>> SQL is much older than DBFCDX.
>>
>> EMG
> 
> David:
> 
>     I'm older than *any* technology around today (except maybe the 
> automobile) :((
> 
>     I 'sampled' SQL many years ago, and found it daunting and 'remote'. 
> That is,  a programmer couldn't get really *close* to the actual data inside 
> the SQL database -- it seems to have a wall around the actual files).  With 
> DBFCDX and its predecessors (DBFNTX, etc),  I feel much closer to the data 
> than with the (frightening) SQL database protocol.
> 
>     Is modern SQL faster than CDX stuff ??
> 
> -Mel Smith 
> 
> 
To all the three old foxes here (four with me, born in 1958).
Are we kidding? No question about the fact that, as stated by EMG, SQL 
technology is maybe older than CDX (ask Larry Ellison...). But in the 
Good Old Times, none with a gram (an ounce for yankees, eh...) of brain 
could have maintained that SQL was more performing than a straight .DBF 
access. The former could have been implemented only in expensive 
datawarehouses with powerful machines, while even a 286 PC was able to 
retrieve data at an astonishing speed in a mere Novell 2.1 network, DOS 
clients, ugh, without the need of expensive hardware resources. (Oh, BTW 
a 286 WAS quite expensive for a small business at the time).
We are talking about 10Mb networks in best cases, I implemented at the 
time 10-net networks, working IIRC at 1 Mb/s. Do you remember?
I remember the Oracle's catalog knocked me down simply with the number 
of zeroes the license costed! Not considering the hardware needed! THAT 
was the reason for Dbase et.al. gained so much momentum, I think. 
Performance vs. cost. And THEN Clipper arrived, eh. THE solution, not 
"a" solution.

So for what kind of reason now, that machines are so astonishing 
powerful, networks shoot data at 100 Mb/sec and we have this wonderful 
(x)Harbour 32-bit all-new stuff, Linux kernel 2.6 and a ton of 
improvements regarding Samba, etc., well, I repeat, for what kind of 
reason a stupid database of 50000/60000 records needs 15 seconds on a 
modern PC to be opened?

Sorry, guys, I don't get it. I missed the train maybe.
I feel like a taxpayer that is asked to pay an extra amount in order to 
have drinking water from his tap (not in California, in Montana)!
There is something that is quite different from things that have been 
told me. I will go the SQL way if necessary, but I am sure I will have a 
number of problems/drawback/messes that way, too. There are no free 
lunches, isn't it?

And sorry for this - likely - waste of your time, it was only a far cry 
from someone in a moment of angry (the moments you have the feeling to 
get real, and what you see doesn't like you at all!).
Have a good programming, men. I promise to keep you informed, but, 
having seen around EMG and DLZC for a while (I was exchanging opinions 
with EMG some 18 years ago in FIDOnet...) I am not really confident I'll 
find out an easy, obvious explanation of this mess and a quick solution 
(other than go back to Clipper, eh eh!...).
Best regards.
Dan

Dear Enrico Maria Giordano:

On Jan 14, 11:00=A0am, "Enrico Maria Giordano"
<e.m.giord...@emagsoftware.it> wrote:
> "Mel Smith" <med_cutout_syn...@aol.com> ha scritto nel messaggionews:7r93=
9pFh4nU1@mid.individual.net...
>
> > =A0 =A0I'm older than *any* technology around today (except
> > maybe the automobile) :((
>
> I'm class 1960, and you?

I remember JFK being shot, but not too much before that.

> > =A0 =A0I 'sampled' SQL many years ago, and found it
> > daunting and 'remote'.  That is, =A0a programmer couldn't
> > get really *close* to the actual data inside the SQL
> > database -- it seems to have a wall around the actual
> > files).

This was by design.  Minimizes data transfer, a necessity then, and a
really good feature now.  Also centralizes data for security, backups,
etc.

> >=A0With DBFCDX and its predecessors (DBFNTX, etc),
> >=A0I feel much closer to the data than with the (frightening)
> > SQL database protocol.
>
> Yes, but that is considered an advantage from the
> security point of view.
>
> > =A0 =A0Is modern SQL faster than CDX stuff ??
>
> Yes and no. It depends. It's not so easy to compare
> those two deeply phylosophycally different
> technologies.

Multitasking always appears faster, where there is need.  Network
communication speeds have increased, but still cannot meet CPU-to-
memory or CPU-to-hard-disk speeds.  SQL must beat DBFCDX, for more
than a handfull of users (or users that are always looking at the same
handfull of records).

David A. Smith

Enrico Maria Giordano ha scritto:
> 
> Yes. I always use latest xHarbour from CVS.

In production? Hm, I am quite conservative in comparison...

> 
> So yours is a different problem, I suppose.

Same environment? Linux, CGI accessing data?
(BTW I tweaked a bit with a couple of Samba parameters, to no avail.)
It seems that the performance degradation came after the CGI 
implementation. But maybe it's not related.

>> And, by the way, how have you managed / planned to solve the problem?
> 
> No, I only alleviated it by reducing unnecessary USE/CLOSE.

Unnecessary... yes, but I feared, among the others, the old 
COMMIT/related problems, and took the habit to close/reopen files many 
years ago, when I found that this was not a performance killer due to 
caching of data by the server.

I started with Clipper working on floppies, and open/close was a pain, 
but when they invented hard disks, all went fast. Are we going back or 
forward? :-)

BTW, Novell was way faster than SAMBA/Linux. Not even comparable.
But I had so many problems with Novell 5, so I ended up with Linux. 
Maybe you tried a Windows server?...

> EMG

Dan

"D.Campagna" <ynnadrebyc@tiscalinet.it> ha scritto nel messaggio 
news:hio0dp$5fn$1@nnrp-beta.newsland.it...

> We are talking about 10Mb networks in best cases, I implemented at the 
> time 10-net networks, working IIRC at 1 Mb/s. Do you remember?

Yes, me too. But I remember 10 Mb (what 10-net stand for?).

> So for what kind of reason now, that machines are so astonishing powerful, 
> networks shoot data at 100 Mb/sec and we have this wonderful (x)Harbour 
> 32-bit all-new stuff, Linux kernel 2.6 and a ton of improvements regarding 
> Samba, etc., well, I repeat, for what kind of reason a stupid database of 
> 50000/60000 records needs 15 seconds on a modern PC to be opened?

That's exactly my thought! There must be something terribly wrong 
somewhere...

> Have a good programming, men. I promise to keep you informed, but, having 
> seen around EMG and DLZC for a while (I was exchanging opinions with EMG 
> some 18 years ago in FIDOnet...)

Gasp! :-)

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

"D.Campagna" <ynnadrebyc@tiscalinet.it> ha scritto nel messaggio 
news:hio24c$6h3$1@nnrp-beta.newsland.it...

> > Yes. I always use latest xHarbour from CVS.
>
> In production? Hm, I am quite conservative in comparison...

Yes. Now that RDDs are rock solid the risks in using xHarbour from CVS are 
close to zero. This is the only way to seriously test it. And if something 
doesn't work as expected I simply switch back to the previous xHarbour 
build.

> > So yours is a different problem, I suppose.
>
> Same environment? Linux, CGI accessing data?

No, Windows peer to peer.

> > No, I only alleviated it by reducing unnecessary USE/CLOSE.
>
> Unnecessary... yes, but I feared, among the others, the old COMMIT/related 
> problems, and took the habit to close/reopen files many years ago, when I 
> found that this was not a performance killer due to caching of data by the 
> server.

I was referring to USE/CLOSE tables for reading.

> I started with Clipper working on floppies, and open/close was a pain, but 
> when they invented hard disks, all went fast. Are we going back or 
> forward? :-)

I really don't know...

> Maybe you tried a Windows server?...

No, just Windows PCs with peer to peer networks.

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

"Enrico Maria Giordano" <e.m.giordano@emagsoftware.it> wrote in message 
news:4b4f5bda$0$823$4fafbaef@reader5.news.tin.it...

> I'm class 1960, and you?
>


    I've been computer programming steadily since 1968 (with Fortran and 
Algol at the University of Alberta) (and I was 36yo *then*), and I'm still 
far behind in technology.   I was sent to university as a Captain in the 
Canadian Artillery for several years.  Exited with a M.Sc. in Computing 
Science  (thesis was " On the Detection of Edges in Pictures" -- front end 
of Computer Graphics).

    ... and now I'm an old (but busy) man


-Mel

"Mel Smith" <med_cutout_syntel@aol.com> ha scritto nel messaggio 
news:7r9n2lFgbgU1@mid.individual.net...

>    I've been computer programming steadily since 1968 (with Fortran and 
> Algol at the University of Alberta) (and I was 36yo *then*), and I'm still 
> far behind in technology.   I was sent to university as a Captain in the 
> Canadian Artillery for several years.  Exited with a M.Sc. in Computing 
> Science  (thesis was " On the Detection of Edges in Pictures" -- front end 
> of Computer Graphics).
>
>    ... and now I'm an old (but busy) man

My sincere compliments!

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

Enrico Maria Giordano ha scritto:

> 
> Yes, me too. But I remember 10 Mb (what 10-net stand for?).
> 
10-Net, called "tennet" in Italy too, was a proprietary network system. 
It had some diffusion at the beginning of the '90 IIRC. There was the 
software package and the network cards. I think I have still a couple in 
the cellar. :-)

> 
> That's exactly my thought! There must be something terribly wrong 
> somewhere...

:-) And let's hope the wrong ones are not us!

>> seen around EMG and DLZC for a while (I was exchanging opinions with EMG 
>> some 18 years ago in FIDOnet...)
> 
> Gasp! :-)

Well, probably it was only 15 yrs ago... 1995 maybe. I should have the 
backups of all the threads i was paying attention to. (I wanted to write 
a book about the first years of the telecommunication craze... maybe one 
day or another I'll do it, when and if I reach the respectable age of 
mr. Mel Smith, with a similar good shape!) :-)

> 
> EMG

> The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
my first "computer" was a ZX-81...

Dan

"D.Campagna" <ynnadrebyc@tiscalinet.it> ha scritto nel messaggio 
news:hipher$22f$1@nnrp-beta.newsland.it...

> 10-Net, called "tennet" in Italy too, was a proprietary network system. It 
> had some diffusion at the beginning of the '90 IIRC.

No, I installed 10-Net in the second half of '80.

> my first "computer" was a ZX-81...

Me too. :-)

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

On Thu, 14 Jan 2010 21:56:23 +0100, "D.Campagna" <ynnadrebyc@tiscalinet.it> wrote:

Hello to all,
Here my little experiences. Sorry, I'm not a professional!
After using NTX, I'm still using DBFCDX. 
In fact, using DBFCDX indexes is very fast also with many 10'000 of records.
But, after changing to a Microsoft servers (Currently SBS2003) everything has become slower!
Prior to Novell, I have had almost the same response time as a local disk.
Now, compare the same PC with the same application is for some 
While .not. Eof() cycle from 2 und 5 times slower. 
I can't give any answer about SQL!
Regards
Otto

>Mel Smith ha scritto:
>> David & Enrico said:
>> 
>>>> Same here.  It is only the recent past technology, where DBFCDX is
>>>> distant past technology.
>>> SQL is much older than DBFCDX.
>>>
>>> EMG
>> 
>> David:
>> 
>>     I'm older than *any* technology around today (except maybe the 
>> automobile) :((
>> 
>>     I 'sampled' SQL many years ago, and found it daunting and 'remote'. 
>> That is,  a programmer couldn't get really *close* to the actual data inside 
>> the SQL database -- it seems to have a wall around the actual files).  With 
>> DBFCDX and its predecessors (DBFNTX, etc),  I feel much closer to the data 
>> than with the (frightening) SQL database protocol.
>> 
>>     Is modern SQL faster than CDX stuff ??
>> 
>> -Mel Smith 
>> 
>> 
>To all the three old foxes here (four with me, born in 1958).
>Are we kidding? No question about the fact that, as stated by EMG, SQL 
>technology is maybe older than CDX (ask Larry Ellison...). But in the 
>Good Old Times, none with a gram (an ounce for yankees, eh...) of brain 
>could have maintained that SQL was more performing than a straight .DBF 
>access. The former could have been implemented only in expensive 
>datawarehouses with powerful machines, while even a 286 PC was able to 
>retrieve data at an astonishing speed in a mere Novell 2.1 network, DOS 
>clients, ugh, without the need of expensive hardware resources. (Oh, BTW 
>a 286 WAS quite expensive for a small business at the time).
>We are talking about 10Mb networks in best cases, I implemented at the 
>time 10-net networks, working IIRC at 1 Mb/s. Do you remember?
>I remember the Oracle's catalog knocked me down simply with the number 
>of zeroes the license costed! Not considering the hardware needed! THAT 
>was the reason for Dbase et.al. gained so much momentum, I think. 
>Performance vs. cost. And THEN Clipper arrived, eh. THE solution, not 
>"a" solution.
>
>So for what kind of reason now, that machines are so astonishing 
>powerful, networks shoot data at 100 Mb/sec and we have this wonderful 
>(x)Harbour 32-bit all-new stuff, Linux kernel 2.6 and a ton of 
>improvements regarding Samba, etc., well, I repeat, for what kind of 
>reason a stupid database of 50000/60000 records needs 15 seconds on a 
>modern PC to be opened?
>
>Sorry, guys, I don't get it. I missed the train maybe.
>I feel like a taxpayer that is asked to pay an extra amount in order to 
>have drinking water from his tap (not in California, in Montana)!
>There is something that is quite different from things that have been 
>told me. I will go the SQL way if necessary, but I am sure I will have a 
>number of problems/drawback/messes that way, too. There are no free 
>lunches, isn't it?
>
>And sorry for this - likely - waste of your time, it was only a far cry 
>from someone in a moment of angry (the moments you have the feeling to 
>get real, and what you see doesn't like you at all!).
>Have a good programming, men. I promise to keep you informed, but, 
>having seen around EMG and DLZC for a while (I was exchanging opinions 
>with EMG some 18 years ago in FIDOnet...) I am not really confident I'll 
>find out an easy, obvious explanation of this mess and a quick solution 
>(other than go back to Clipper, eh eh!...).
>Best regards.
>Dan

Otto Haldi ha scritto:
> On Thu, 14 Jan 2010 21:56:23 +0100, "D.Campagna" <ynnadrebyc@tiscalinet.it> wrote:
> 
> Hello to all,
> Here my little experiences. Sorry, I'm not a professional!
> After using NTX, I'm still using DBFCDX. 
> In fact, using DBFCDX indexes is very fast also with many 10'000 of records.

CDX indexes are faster and more reliable, mainly with regard to memo 
fields. I think not many people here still use .NTX. Nobody perhaps.

> But, after changing to a Microsoft servers (Currently SBS2003) everything has become slower!
> Prior to Novell, I have had almost the same response time as a local disk.
> Now, compare the same PC with the same application is for some 
> While .not. Eof() cycle from 2 und 5 times slower. 

Novell was the fastest network system ever seen for PC's. Unfortunately, 
version 5 had so many quirks and errors (from the client side: and 
specifically with Clipper applications running under Windows 95 I had 
very hard times because of a bug in the Novell client in his first 
releases) and a ridicolous price to upgrade. So I moved to Linux/Samba.

People noticed a performance degradation, but not so dramatic to worry 
about. And it was free, after all.
But now things are getting worse... and it's not going to make me 
happier just to know that I'm not alone in the mess!

> Regards
> Otto

Dan

Hi All
    Well I guess I am another old one. Started with Novell & RXnet (I think) 
in the middle 80's. I am still using Novell (version 6) - primarily just 
file sharing, now using GB switches & cards, & still NTX files. I can 
generate reports quicker off the server then I can off my local SATA 7200 
rpm hard drive. I also have one install using BSD & Samba, which has 
performed well.
    One previous cause of file opening slowly was having a large number of 
deleted records at the start of the file or index order.

Regards
Harold



"D.Campagna" <retsamtsop@danielecampagna.co.cc> wrote in message 
news:hiq1lj$ckk$1@nnrp-beta.newsland.it...
> Otto Haldi ha scritto:
>> On Thu, 14 Jan 2010 21:56:23 +0100, "D.Campagna" 
>> <ynnadrebyc@tiscalinet.it> wrote:
>>
>> Hello to all,
>> Here my little experiences. Sorry, I'm not a professional!
>> After using NTX, I'm still using DBFCDX. In fact, using DBFCDX indexes is 
>> very fast also with many 10'000 of records.
>
> CDX indexes are faster and more reliable, mainly with regard to memo 
> fields. I think not many people here still use .NTX. Nobody perhaps.
>
>> But, after changing to a Microsoft servers (Currently SBS2003) everything 
>> has become slower!
>> Prior to Novell, I have had almost the same response time as a local 
>> disk.
>> Now, compare the same PC with the same application is for some While 
>> .not. Eof() cycle from 2 und 5 times slower.
>
> Novell was the fastest network system ever seen for PC's. Unfortunately, 
> version 5 had so many quirks and errors (from the client side: and 
> specifically with Clipper applications running under Windows 95 I had very 
> hard times because of a bug in the Novell client in his first releases) 
> and a ridicolous price to upgrade. So I moved to Linux/Samba.
>
> People noticed a performance degradation, but not so dramatic to worry 
> about. And it was free, after all.
> But now things are getting worse... and it's not going to make me happier 
> just to know that I'm not alone in the mess!
>
>> Regards
>> Otto
>
> Dan 

Il Fri, 15 Jan 2010 12:56:37 -0500, H Davis ha scritto:

> Hi All
>     Well I guess I am another old one. Started with Novell & RXnet (I
>     think)
> in the middle 80's. I am still using Novell (version 6) - primarily just
> file sharing, now using GB switches & cards, & still NTX files. I can
> generate reports quicker off the server then I can off my local SATA
> 7200 rpm hard drive. I also have one install using BSD & Samba, which
> has performed well.
>     One previous cause of file opening slowly was having a large number
>     of
> deleted records at the start of the file or index order.
> 
> Regards
> Harold

Hi Harold

Hust one information: if you open the same database with two different 
computer at the same time (so DBF shared and 2 or more PC connected to 
it), do you have a strong degradation in performance or not?
I'm just trying to understand if this problem I have is related to the 
server or to the client. I remember that this problem were present since 
WXP, (no problem with W98): just to see if well remember

-- 
Ciao, Alessandro

P.S.: I would be surprised if there were some young people using clipper 
and successors!

"Alessandro Antonangeli" <alcati.r3m0v3th1s@alcati.com> ha scritto nel 
messaggio news:his5he$s74$1@speranza.aioe.org...

> Hust one information: if you open the same database with two different
> computer at the same time (so DBF shared and 2 or more PC connected to
> it), do you have a strong degradation in performance or not?

I don't know. I made all my tests using only one user.

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

Enrico Maria Giordano ha scritto:
> "Alessandro Antonangeli" <alcati.r3m0v3th1s@alcati.com> ha scritto nel 
> messaggio news:his5he$s74$1@speranza.aioe.org...
> 
>> Hust one information: if you open the same database with two different
>> computer at the same time (so DBF shared and 2 or more PC connected to
>> it), do you have a strong degradation in performance or not?
> 
> I don't know. I made all my tests using only one user.
> 
> EMG
We have in this environment 5 users and a dozen of 'externals' accessing 
via the web interface, but since I have not (still) put up an access 
tracing system, I really don't know for sure if someone is accessing the 
system from outside (= if the Linux cgi is operating) and when.

Until a certain point, no performance degradation was perceived, even 
with 5 users busy working on the tables (in the intranet). And that's 
what should be. Normal. As expected. Of course, no comparison with local 
access. We are on a lan, eh. A bit slower.
What we are experiencing, is a suddenly performance degradation that is 
simply causing me to be called and asked if the whole thing is now worth 
the effort to put it up. Figure it out! I was the main endorser of it. 
I'm between a rock and a hard place, as to say. :-(

To EMG, Antonangeli and Davis (and of course to everyone interested in 
the matter):
Maany thanks for the answering. I found them illuminating. I am trying 
to put up a testcase to understand if it's a client (I lean on it) or a 
server problem. I have the original executable in Clipper that does 
(more or less) the same things the new xHarbour-compiled program does.
I say, I can test the exactly same program in different environments and 
compiled with different compilers. Basically, the new xHarbour+wvw 
program differs only in the user interface.
At present, the 3 executables seem to show the same slow behaviour. I 
compiled (after some difficulties) the program with xHarbour 1.21 and 
the newly-adapted wvw, and it seems to be even more slow than the 
original 0.99+wvw and the even-more-original executable clipper 5.3 PURE.
I think I am really one out of a zillion that can perform those tests, 
(because of my laziness in upgrading the original program, huh?), and 
the subsequent substantial equivalence of the ancient clipper 
executable, the recent xharbour 0.99 and the same source recompiled 
(with no changes at all) with xharbour 1.21.

So now I'll try to connect an 'ancient' client running w98 and execute 
the program from it. If it's faster, it's definitely a client problem.
Stay tuned if you are interested in the testing!
And I'll give a try to a client running Linux and executing the program 
under Wine. Why not? Let's have fun!
I will post the code (a reduced sample at least) and the results.
Let's get ride of it.
Dan

"Otto Haldi" <ohaldi@chateau.ch> ha scritto nel messaggio 
news:j3n0l5trou7sktlgh8a88p1t18hhf2p47j@4ax.com...

> While .not. Eof() cycle from 2 und 5 times slower.

I just tried such a cycle and not found great difference between local hard 
disk and remote hard disk (wireless network with my Vista notebook). I don't 
understand...

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

"D.Campagna" <retsamtsop@danielecampagna.co.cc> ha scritto nel messaggio 
news:hitfpi$bhs$1@nnrp-beta.newsland.it...

> So now I'll try to connect an 'ancient' client running w98 and execute the 
> program from it. If it's faster, it's definitely a client problem.
> Stay tuned if you are interested in the testing!

I'm really interesting!

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

Enrico Maria Giordano ha scritto:
> "Otto Haldi" <ohaldi@chateau.ch> ha scritto nel messaggio 
> news:j3n0l5trou7sktlgh8a88p1t18hhf2p47j@4ax.com...
> 
>> While .not. Eof() cycle from 2 und 5 times slower.
> 
> I just tried such a cycle and not found great difference between local hard 
> disk and remote hard disk (wireless network with my Vista notebook). I don't 
> understand...
> 
I heard a lot of (meaningful) warnings about wireless access to DBF's.
It's generally discouraged as it can cause data corruption...
But we really need to post code used for testing and discuss it if we 
want to get some real answer.
[ot]
Hey EMG, I know for sure you are really very qualified, you have deep 
insight of Clipper, Fivewin, C programming, and God knows what else.
It seems you are on the list of contributors of xharbour and you in fact 
did make some contribution, too.
Your mild-mannered and quite dubious comments come as a surprise. 15 
years ago you was very sure about you, and you offered answers 
(sometimes a bit.. ahem.. let's say over-conscious of your knowledge...) 
to everyone had a doubt.
What's happening? I could have guessed you would have come out with some 
  hints ( and maybe some bitter comments :-). )
Your low-profile comments are really scaring me.
[end ot]
Dan

"D.Campagna" <retsamtsop@danielecampagna.co.cc> ha scritto nel messaggio 
news:hitja0$di3$1@nnrp-beta.newsland.it...

> > I just tried such a cycle and not found great difference between local 
> > hard disk and remote hard disk (wireless network with my Vista 
> > notebook). I don't understand...
>
> I heard a lot of (meaningful) warnings about wireless access to DBF's.
> It's generally discouraged as it can cause data corruption...

I'm using this wireless network I have here at home only for testing 
purpose.

> But we really need to post code used for testing and discuss it if we want 
> to get some real answer.

Here it is:

REQUEST DBFCDX


FUNCTION MAIN()

    LOCAL nSec

    LOCAL cCli

    RDDSETDEFAULT( "DBFCDX" )

    USE E:\EMAGFW\GESR\CORSE

    nSec = SECONDS()

    WHILE !EOF()
        cCli = FIELD -> cliente
        SKIP
    ENDDO

    ? "Assign and skip", SECONDS() - nSec

    nSec = SECONDS()

    INDEX ON DTOS( FIELD -> data ) + STR( FIELD -> numero, 3 ) TO CORSE FOR 
!EMPTY( FIELD -> numero ) TEMPORARY

    ? "Create index", SECONDS() - nSec

    nSec = SECONDS()

    WHILE !EOF()
        cCli = FIELD -> cliente
        SKIP
    ENDDO

    ? "Assign and skip with index", SECONDS() - nSec

    CLOSE

    FERASE( "CORSE.CDX" )

    INKEY( 0 )

    RETURN NIL

> Your mild-mannered and quite dubious comments come as a surprise. 15 years 
> ago you was very sure about you, and you offered answers (sometimes a 
> bit.. ahem.. let's say over-conscious of your knowledge...) to everyone 
> had a doubt.
> What's happening? I could have guessed you would have come out with some 
> hints ( and maybe some bitter comments :-). )
> Your low-profile comments are really scaring me.

I simply can't understand what's going on. Experiments show contraddictory 
results and I really don't know what is happening. If you can confirm that 
the problem is the new Windows versions then I will try to imagine some sort 
of solution. Otherwise I'm simply lost for now.

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

Enrico Maria Giordano ha scritto:

> 
>> But we really need to post code used for testing and discuss it if we want 
>> to get some real answer.
> 
> Here it is:
> 
> REQUEST DBFCDX
> 
> 
> FUNCTION MAIN()
> 
>     LOCAL nSec
> 
>     LOCAL cCli
> 
>     RDDSETDEFAULT( "DBFCDX" )
> 
>     USE E:\EMAGFW\GESR\CORSE
> 
>     nSec = SECONDS()
> 
>     WHILE !EOF()
>         cCli = FIELD -> cliente
>         SKIP
>     ENDDO
> 
>     ? "Assign and skip", SECONDS() - nSec
> 
>     nSec = SECONDS()
> 
>     INDEX ON DTOS( FIELD -> data ) + STR( FIELD -> numero, 3 ) TO CORSE FOR 
> !EMPTY( FIELD -> numero ) TEMPORARY
> 
>     ? "Create index", SECONDS() - nSec
> 
>     nSec = SECONDS()
> 
>     WHILE !EOF()
>         cCli = FIELD -> cliente
>         SKIP
>     ENDDO
> 
>     ? "Assign and skip with index", SECONDS() - nSec
> 
>     CLOSE
> 
>     FERASE( "CORSE.CDX" )
> 
>     INKEY( 0 )
> 
>     RETURN NIL
ACK.
[italian ON - if translation needed please ask]
Si ma io ho notato il rallentamento sul sempice open, qui il tempo � 
misurato su operazioni susseguenti.
Fino a luned� niente test, sono in giro. Il vero applicativo apre 
diverse tabelle e sembra che il ritardo sia in fase di USE, mentre dopo 
tutto fila liscio.
Ho in mente anche di provare a scrivere una funzioncella che apre i DBF 
con FOPEN, questo bypassa i RDD. far� qualcosa del genere:
t:=seconds()
use clienti
(misura quanro tempo � passato)
use
[aridanga]
t=seconds()
use clienti index clienti
[misura t]
use
a qesto punto test con fopen()
ecc.
Naturalmente prima su XP, poi con W98 e poi (eventualmente) con Wine su 
Linux.

Ti faccio sapere appena ho qualche lume.
Tempo fa ho visto che postavi relativamente a SQL.
Hai qualche indicazione di migliori performances?
Intanto grazie e a presto.

> EMG

Dan

"Enrico Maria Giordano" <e.m.giordano@emagsoftware.it> ha scritto nel 
messaggio news:4b525369$0$818$4fafbaef@reader5.news.tin.it...

>    USE E:\EMAGFW\GESR\CORSE

I made a very very stupid error! I used a local DBF! These are the results 
with local and remote DBF:

Local

0.28
0.38
0.25

Remote

125
15
131

So my conclusion is that my network is incredibly slow! Ok, it's a wireless 
network but it seems too slow anyway.

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

"D.Campagna" <retsamtsop@danielecampagna.co.cc> ha scritto nel messaggio 
news:hitr8a$hnp$1@nnrp-beta.newsland.it...

> Si ma io ho notato il rallentamento sul sempice open, qui il tempo � 
> misurato su operazioni susseguenti.

Here is an Open test:

REQUEST DBFCDX


FUNCTION MAIN()

    LOCAL nSec

    LOCAL i

    RDDSETDEFAULT( "DBFCDX" )

    nSec = SECONDS()

    FOR i = 1 TO 100
        USE CORSE
    NEXT

    ? "Open", SECONDS() - nSec

    INKEY( 0 )

    RETURN NIL

Results:

Local

0.02

Remote

2.25

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

"D.Campagna" <retsamtsop@danielecampagna.co.cc> ha scritto nel messaggio 
news:hitr8a$hnp$1@nnrp-beta.newsland.it...

> Ti faccio sapere appena ho qualche lume.

Ok, grazie.

> Tempo fa ho visto che postavi relativamente a SQL.
> Hai qualche indicazione di migliori performances?

Purtroppo no. Non ho mai fatto test comparativi seri.

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

Hi

First some tests I did (but keep in mind I have a WXP on a Virtual 
machine).

Open 1000 times a DBF (in shared mode)
Locally 0,16 seconds
On a shared drive 3,82 seconds (25x time slower)
On a shared drive, but the file opened with another process, 2,64 seconds 
(16x time slower, it seems the SO is helped from beeing already opened 
from the other process)

If another process has the same file opened from ANOTHER machine:
- file opening time is 52x slower than local, 2x slower than on lan
- field write is 120x time slower than local, 100x time slower than on lan
- field reading is 100x time slower than local, 23x time slower than on 
lan

At the end, the code (mostly from EMG) used for the test. I created a dbf 
with 3 field with 200 records

I really do not think (not tested) than the records number can have 
influence on file opening

One possible reason for your slow performance is that some CGI process 
leaves DBF open. I can suggest to check if:

1) the problems come out when a CGI application have run
2) if some CGI application is still alive

HTH

****CODE****


REQUEST DBFCDX


FUNCTION MAIN()

    LOCAL nSec
LOCAL i
    LOCAL cCli

    RDDSETDEFAULT( "DBFCDX" )

    nSec = SECONDS()
   For i=1 to 1000
       USE Prova SHARED
   NEXT
    ? "Apertura DBF", SECONDS() - nSec

    nSec = SECONDS()
       USE Prova SHARED
   For i=1 to 10
	GO TOP
    WHILE !EOF()
		RLOCK()
        FIELD -> cliente:=STR(RECNO())
        FIELD -> Numero:=RECNO()
        FIELD -> Data:=Date()
        SKIP
    ENDDO
	NEXT
    ? "Assegnamento campi", SECONDS() - nSec
    nSec = SECONDS()

   For i=1 to 100
       USE Prova SHARED
	GO TOP
    WHILE !EOF()
        cCli = FIELD -> cliente
        SKIP
    ENDDO
   NEXT

    ? "Assign and skip", SECONDS() - nSec

****END CODE****
-- 
Ciao, Alessandro

Alessandro (and everyone else),

>Open 1000 times a DBF (in shared mode)
>Locally 0,16 seconds
>On a shared drive 3,82 seconds (25x time slower)
>On a shared drive, but the file opened with another process, 2,64 seconds 
>(16x time slower, it seems the SO is helped from beeing already opened 
>from the other process)

Have you considered the usual registry tweaks like, for example,
turning off oplocks? Nick Ramsay's NT Networking page has been
"required reading" for Clipper programmers for many years, and the
solutions apply equally for running xHarbour apps on network shares.
Take a look here:

http://www.witzendcs.co.uk/nt_networking.html

For further info, follow the links on the page, and/or search the web
and the MS knowledgebase for each setting.

Regards,
Klas

-------
klas dot engwall at engwall dot com

http://www.engwall.com/clipper/

The LFN Library for Clipper
The LanMan Library for Clipper
The NFPAT1A Timeslice release patch for the Nanforum Toolkit

Dear Enrico Maria Giordano:

On Jan 17, 5:20=A0am, "Enrico Maria Giordano"
<e.m.giord...@emagsoftware.it> wrote:
> "Enrico Maria Giordano" <e.m.giord...@emagsoftware.it> ha scritto nel
> messaggionews:4b525369$0$818$4fafbaef@reader5.news.tin.it...
>
> > =A0 =A0USE E:\EMAGFW\GESR\CORSE
>
> I made a very very stupid error! I used a local DBF!
> These are the results with local and remote DBF:
>
> Local
>
> 0.28
> 0.38
> 0.25
>
> Remote
>
> 125
> 15
> 131
>
> So my conclusion is that my network is incredibly
> slow! Ok, it's a wireless network but it seems too
> slow anyway.

Wireless and DBF should always be considered to be mutually
exclusive.  Too much risk of damaging the file.

How do you open the file, by "D:\path\filename" or "\\server\path
\filename"?

David A. Smith

"dlzc" <dlzc1@cox.net> ha scritto nel messaggio 
news:78426522-f085-452e-a783-8b137f76ba9b@22g2000yqr.googlegroups.com...

> Wireless and DBF should always be considered to be mutually
> exclusive.  Too much risk of damaging the file.

As I already said, I'm using this network only for testing purpose.

> How do you open the file, by "D:\path\filename" or "\\server\path
> \filename"?

The former. But I already tried the latter and find no difference.

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

> xharbour 0.99

Have you tested this with Harbour instead of xHarbour ?
The RDD guys are not playing wth xHarbour anymore...

-- 
Maur�cio Faria

"Maur�cio Ventura Faria" <mauricio@farmacia.com.br> ha scritto nel messaggio 
news:hj1mtc$uq5$1@news.eternal-september.org...

> Have you tested this with Harbour instead of xHarbour ?

Yes, I did. No difference at all.

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

On Jan 18, 1:44=A0am, "Enrico Maria Giordano"
<e.m.giord...@emagsoftware.it> wrote:
> "dlzc" <dl...@cox.net> ha scritto nel messaggionews:78426522-f085-452e-a7=
83-8b137f76ba9b@22g2000yqr.googlegroups.com...
>
> > Wireless and DBF should always be considered
> > to be mutually exclusive. =A0Too much risk of
> > damaging the file.
>
> As I already said, I'm using this network only for
> testing purpose.

Then I will quit whining.

> > How do you open the file, by "D:\path\filename"
> > or "\\server\path\filename"?
>
> The former. But I already tried the latter and find
> no difference.

What OpSys are you running this on?  I suspect M$ of fine tuning
differences to try and force people off of *nix, due to poor
performance with their desktop machines.  You may have already stated
this here.  I think someone said the data files resided on Linux.

David A. Smith

On Jan 15, 11:56=A0am, "H Davis" <hbda...@sympatico.ca> wrote:
> Hi All
> =A0 =A0 Well I guess I am another old one. Started with Novell & RXnet (I=
 think)
> in the middle 80's. I am still using Novell (version 6) - primarily just
> file sharing, now using GB switches & cards, & still NTX files. I can
> generate reports quicker off the server then I can off my local SATA 7200
> rpm hard drive. I also have one install using BSD & Samba, which has
> performed well.
> =A0 =A0 One previous cause of file opening slowly was having a large numb=
er of
> deleted records at the start of the file or index order.
>
> Regards
> Harold
>
> "D.Campagna" <retsamt...@danielecampagna.co.cc> wrote in message
>
> news:hiq1lj$ckk$1@nnrp-beta.newsland.it...
>
>
>
> > Otto Haldi ha scritto:
> >> On Thu, 14 Jan 2010 21:56:23 +0100, "D.Campagna"
> >> <ynnadre...@tiscalinet.it> wrote:
>
> >> Hello to all,
> >> Here my little experiences. Sorry, I'm not a professional!
> >> After using NTX, I'm still using DBFCDX. In fact, using DBFCDX indexes=
 is
> >> very fast also with many 10'000 of records.
>
> > CDX indexes are faster and more reliable, mainly with regard to memo
> > fields. I think not many people here still use .NTX. Nobody perhaps.
>
> >> But, after changing to a Microsoft servers (Currently SBS2003) everyth=
ing
> >> has become slower!
> >> Prior to Novell, I have had almost the same response time as a local
> >> disk.
> >> Now, compare the same PC with the same application is for some While
> >> .not. Eof() cycle from 2 und 5 times slower.
>
> > Novell was the fastest network system ever seen for PC's. Unfortunately=
,
> > version 5 had so many quirks and errors (from the client side: and
> > specifically with Clipper applications running under Windows 95 I had v=
ery
> > hard times because of a bug in the Novell client in his first releases)
> > and a ridicolous price to upgrade. So I moved to Linux/Samba.
>
> > People noticed a performance degradation, but not so dramatic to worry
> > about. And it was free, after all.
> > But now things are getting worse... and it's not going to make me happi=
er
> > just to know that I'm not alone in the mess!
>
> >> Regards
> >> Otto
>
> > Dan- Hide quoted text -
>
> - Show quoted text -

Duplicate key values is a big slowdown. Creating unique keys and
packing files can provide a big performance boost.

"dlzc" <dlzc1@cox.net> ha scritto nel messaggio 
news:c38ba18b-e0b3-40ea-89b0-6201dd69dc77@21g2000yqj.googlegroups.com...

> What OpSys are you running this on?

The shared drive is on Windows Vista, the client is Windows XP.

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

On Jan 18, 10:13=A0am, "Enrico Maria Giordano"
<e.m.giord...@emagsoftware.it> wrote:
> "dlzc" <dl...@cox.net> ha scritto nel messaggionews:c38ba18b-e0b3-40ea-89=
b0-6201dd69dc77@21g2000yqj.googlegroups.com...
>
> > What OpSys are you running this on?
>
> The shared drive is on Windows Vista, the client
> is Windows XP.

I find a huge number of complaints on M$ website relating to the
slowness of Vista for "discrete file operations" (copy, delete,
rename).  Presumably this would affect you with even an open.
Something to do with recreating thumbnails, maybe.

But did you indicate that this slowness started with a particular
revision of xHarbour?

David A. Smith

"dlzc" <dlzc1@cox.net> ha scritto nel messaggio 
news:fd4f09ef-3d49-4c9d-8bde-ca70dc2d8f9b@j14g2000yqm.googlegroups.com...

> I find a huge number of complaints on M$ website relating to the
> slowness of Vista for "discrete file operations" (copy, delete,
> rename).  Presumably this would affect you with even an open.

I know, but this is only a test. The real slowness has been found by my 
clients in their PC with various configurations.

> But did you indicate that this slowness started with a particular
> revision of xHarbour?

I'm not sure of anything anymore... :-(

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

Enrico Maria Giordano wrote:
> "dlzc" <dlzc1@cox.net> ha scritto nel messaggio 
> news:c38ba18b-e0b3-40ea-89b0-6201dd69dc77@21g2000yqj.googlegroups.com...
> 
>> What OpSys are you running this on?
> 
> The shared drive is on Windows Vista, the client is Windows XP.
> 
> EMG
> 
> --
> EMAG Software Homepage:     http://www.emagsoftware.it
> The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
> The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
> The EMG Music page:         http://www.emagsoftware.it/emgmusic 
> 
> 

Have you turned off the opportunistic locking?
With Vista you might not be able to turn it off.
Check this: http://support.microsoft.com/kb/296264

"GDFord" <gale.ford@wwrowland.com> ha scritto nel messaggio 
news:hj2jrc$fba$1@news.eternal-september.org...

> Have you turned off the opportunistic locking?
> With Vista you might not be able to turn it off.
> Check this: http://support.microsoft.com/kb/296264

My friends, you missed the point: my test environment has nothing to do with 
the problem we are discussing.

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

Dear D.Campagna:

On Jan 13, 3:18=A0pm, "D.Campagna" <ynnadre...@tiscalinet.it> wrote:
> Hi all, I have a problem that is really puzzling me
> and annoying my customers, and I am not sure
> even on what to investigate first. Any hint will be
> appreciated.

Can you give us an update on this?

Do you access the remote resources via "drive:/path/filename" or "\
\servername\path\filename"?

What is the status of "opportunistic locking"?  (Note that this should
have no effect on opening a file, but on moving about in it.)

David A. Smith

dlzc ha scritto:
> Dear D.Campagna:
> 
> On Jan 13, 3:18 pm, "D.Campagna" <ynnadre...@tiscalinet.it> wrote:
>> Hi all, I have a problem that is really puzzling me
>> and annoying my customers, and I am not sure
>> even on what to investigate first. Any hint will be
>> appreciated.

Here I am with first results of testing.
Sorry for i'm late, I had some drawbacks unrelated with this argument.

> 
> Can you give us an update on this?
> 
> Do you access the remote resources via "drive:/path/filename" or "\
> \servername\path\filename"?

Good point, but I kept things very dumb. Since the application resides 
in the same directory of data, it accesses the tables via their name 
only, without any path.
Now, strange enough, I tried to open the files with DBU (recompiled with 
Harbour, and recompiled in plain old Clipper 5.3, with the only 
modification regarding DBFCDX, and it has the expected speed. The file 
opening takes an instant.

True is, my program does not perform a simple USE. All file opening is 
managed by a function (sga(), means Set/Get Area...), and manages to 
open, reopen, set the RDD, indexes, exclusive or shared etc. (refer to 
dbusearea... is similar)
I can post the source, but since it's rather ugly (ehm...) I'll give you 
at first the results and the way I got them.

Environment:
Server Linux (Zenwalk), kernel 2.6.25.4, Samba is 3.2.3
A small standard network with a couple of switch 100 Mb/sec
PC 1, Celeron 1.4 Ghz with XP Home SP3, 384 Mb RAM - runs Avast 
antivirus (if it matters)
PC 2, Athlon 64 3500+, 2Gb RAM, XP Professional SP2 (not updated) - runs 
Avast too and a bunch of resident programs (Daemon tools, Messenger 
etc.) To test Wine it runs Slackware 12.
PC 3, an old K6/500 running Windows 98 (Yes, I know... no comment 
please...) no antivirus at all.
PC 4: a Duron 1.1 Ghz with 512 Mb RAM, running Linux Zenwalk or Windows 
XP Professional SP2, always as a server.
> 
> What is the status of "opportunistic locking"?  (Note that this should
> have no effect on opening a file, but on moving about in it.)
> 
> David A. Smith
I didn't (still) touch any parameter in the win registry, nor set any 
dbflockscheme. All plain vanilla.

Method of testing:
I did not put up a test program, rather added the following code to my 
existing application:
// put up a flag
log_op:=.t.

Added the following before and after file operations:
if log_op
    reglog(<cmessage>)
endif

function reglog is:

function reglog(ctxt)
local curf:=select()
public hlf:=0
if .not. file ("logfile.dbf")
 
dbcreate("logfile",{{"data","D",10,0},{"ora","C",8,0},{"user","N",2,0},{"oper","C",60,0},{"secondi","N",8,2},{"diff","N",8,2}})
endif
if hlf=0
    hlf=sga("logfile") // means more or less dbusearea, returns area 
number (select())
else
    select(hlf)
    if lower(alias())!="logfile"
      hlf=sga("logfile")
      reglog("Reopen logfile")
    endif
endif
pvsec=secondi
cursec=seconds()
append blank
replace data with date()
replace ora with time()
replace user with nodeid
replace oper with ctxt
replace secondi with cursec
replace diff with cursec-pvsec
select(curf)
return nil

This function creates "logfile" and keeps it opened, then adds a record 
with the timings.
The application is compiled with xHarbour latest cvs and WVW. RDD is 
CDX. All files are opened in shared mode.
Files are  varying in size from very small to 2 Mb. Real application 
(test will follow as soon as I'll put my hands on the REAL environment) 
has files about 40 Mb wide. And the problems.

Test 1
------
only one pc accesses data (pc 1) win->server Linux
open a small file 0,02
open a 165K file 4,19
open a small file 0,01
open file 1,2 Mb + 2 indexes 64 Kb and 25 Kb: 0,04
opens 2 more files quite small: 0,06
open file 2 Mb, (with 386K .FPT) + 5 indexes:  0,06
Set relation to some files: 0,01/not measured (0,00)
Some other file opened: 0,11

The second PC does the same operations, while files are in use by the 
first one  (pc 2) win->server Linux
open a small file 0,01
open a 165K file 2,57
open a small file 0,02
open file 1,2 Mb + 2 indexes 64 Kb and 25 Kb: 0,06
opens 2 more files quite small: 0,08
open file 2 Mb, (with 386K .FPT) + 5 indexes:  0,11
Set relation to some file: 0,02
Some other file opened: 0,14

Note to test 1
--------------
Reverting the order, say the Celeron-powered enters as second, shows 
that the more powerful the pc, the fastest the access. But we are 
talking of tenth of seconds.
I've also not been able to access from tghe Win98 PC the share, after a 
bit of tinkering I gave up. Dunno. Samba 3.xx maybe needs some 
configuration to accord with such an ancient thing.

Now let's change the server, PC 4 acts now as a server under Windows XP 
professional SP2 not upgraded.

Test 2
------
Pc 1 accesses data (WinXP->server WinXP)
open a small file 0,03
open a 165K file n.d.
open a small file 0,01
open file 1,2 Mb + 2 indexes 64 Kb and 25 Kb: 0,08
opens 2 more files quite small: 0,10
open file 2 Mb, (with 386K .FPT) + 5 indexes:  0,11
Set relation to some files: 0,01
Some other file opened: 0,14

(pc 2) winXP->server WinXP
open a small file 0,03
open a 165K file 4,57
open a small file 0,01
open file 1,2 Mb + 2 indexes 64 Kb and 25 Kb: 0,09
opens 2 more files quite small: 0,11
open file 2 Mb, (with 386K .FPT) + 5 indexes:  0,11
Set relation to some file: 0,02
Some other file opened: 0,19

Pc3 (note: with a 10 Mb/sec network card) Win98->server WinXP
open a small file 0,00 (not measurable)
open a 165K file 4,78
open a small file 0,00 (not measurable)
open file 1,2 Mb + 2 indexes 64 Kb and 25 Kb: 0,06
opens 2 more files quite small: 0,11
open file 2 Mb, (with 386K .FPT) + 5 indexes:  0,16
Set relation to some file: 0,00 (not measurable)
Some other file opened: 0,22

And so on.
Conclusions:
In my network, everything seems to work smoothly, at least for 3 users. 
I tried to start some other sessions from the XP clients with enough 
resources, and I did not experienced significant lags. A note must be 
given: the old Win98 machine, with an obsolete hardware and a network 
card only able to 10 Mb/sec (but it is the legendary 3Com 
double-buffering network card...) acts as new powerful machines. Gosh!

But now I'll try the real environment, where things are going wrong. 
Maybe the antivirus? They have Panda and McAfee, while I have Avast or 
nothing at all.
No need to test Wine for now, xHarbour here runs as expected, IMO.
Stay tuned and please comment, I'm not sure the way I made the test is 
agreeded/acceptable.
D. Campagna

H Davis ha scritto:

>     One previous cause of file opening slowly was having a large number of 
> deleted records at the start of the file or index order.
> 
> Regards
> Harold

Right, and if this is the problem I am a bit uncomfortable, in fact the 
way my program manages records never needs a pack, they are deleted, 
marked as 'available' and reused when new data are added. Uh-oh.

Dan

Klas Engwall ha scritto:

> 
> Have you considered the usual registry tweaks like, for example,
> turning off oplocks? Nick Ramsay's NT Networking page has been
> "required reading" for Clipper programmers for many years, and the
> solutions apply equally for running xHarbour apps on network shares.
> Take a look here:
> 
> http://www.witzendcs.co.uk/nt_networking.html
> 
> For further info, follow the links on the page, and/or search the web
> and the MS knowledgebase for each setting.
> 
> Regards,
> Klas
> 

Many thanks Klas - bookmarked.
Dan

Dear D.Campagna:

On Jan 19, 5:55=A0am, "D.Campagna" <retsamt...@danielecampagna.co.cc>
wrote:
> dlzc ha scritto:
> > On Jan 13, 3:18 pm, "D.Campagna" <ynnadre...@tiscalinet.it> wrote:
> >> Hi all, I have a problem that is really puzzling me
> >> and annoying my customers, and I am not sure
> >> even on what to investigate first. Any hint will be
> >> appreciated.
>
> Here I am with first results of testing.
> Sorry for i'm late, I had some drawbacks unrelated
> with this argument.

Surely not an argument...

> > Can you give us an update on this?
>
> > Do you access the remote resources via
> > "drive:/path/filename" or "\\servername\path\filename"?
>
> Good point, but I kept things very dumb. Since the
> application resides in the same directory of data,
> it accesses the tables via their name only, without
> any path.

Not good enough.  Can you display / record drive and path information
for the files from inside your application?

> Now, strange enough, I tried to open the files with
> DBU (recompiled with Harbour, and recompiled in
> plain old Clipper 5.3, with the only modification
> regarding DBFCDX, and it has the expected speed.
> The file opening takes an instant.
>
> True is, my program does not perform a simple
> USE. All file opening is managed by a function
> (sga(), means Set/Get Area...), and manages to
> open, reopen, set the RDD, indexes, exclusive or
> shared etc. (refer to dbusearea... is similar)
> I can post the source, but since it's rather ugly
> (ehm...) I'll give you at first the results and the way
> I got them.

....
I'd suggest you parse your SGA function, and record times for each
step within it.

> Conclusions:
> In my network, everything seems to work smoothly,
> at least for 3 users.  I tried to start some other
> sessions from the XP clients with enough
> resources, and I did not experienced significant lags.
> A note must be given: the old Win98 machine, with
> an obsolete hardware and a network card only able
> to 10 Mb/sec (but it is the legendary 3Com
> double-buffering network card...) acts as new powerful
> machines. Gosh!

Yes, "advancement" as the marketers want to tell us is surely NOT
improvement.  But then, I tend to buy based on "value" rather than
absolute specs...

> But now I'll try the real environment, where things
> are going wrong.  Maybe the antivirus? They have
> Panda and McAfee, while I have Avast or nothing
> at all.

I use ClamWin.  AVG had to die because it interfered with gaming.

> No need to test Wine for now, xHarbour here runs
> as expected, IMO.  Stay tuned and please
> comment, I'm not sure the way I made the test is
> agreeded/acceptable.

You are making this more complex, clouding the issue in my opinion.
Clipper apps work via NTVDM, and do so via an assigned drive letter
for a network resource.  I suspect your DBU variant in xHarbour *also*
access the files via a drive letter assignment (unless you seriously
modified its code).

Map a drive letter to the remote resource, and try accessing the files
that way with xHarbour.  Just for grins.  Please.  This is a known and
documented slowdown for file access in Windoze.

David A. Smith

D.Campagna wrote:

> Here I am with first results of testing.
> Sorry for i'm late, I had some drawbacks unrelated with this argument.

I'm sorry I have not really followed the details of this but:

 > But now I'll try the real environment, where things are going wrong. Maybe
 > the antivirus? They have Panda and McAfee, while I have Avast or nothing at
 > all.

I have found McAfee to be a real dog about opening files, if you can switch both 
of these for a short test I would do it.

Regards
Alex

dlzc ha scritto:
> Dear D.Campagna:
....
Quick update (before to go through any other argument):
A file has 10000+ records deleted just at the beginning. It takes 9,32 
seconds to open. Gotcha!
Other tables are opened with timing comparable to the previous test.
It seems that the whole thing was a well-documented issue. Oops.
Now I'll pack the file, rebuild indexes and see what happens.

Dan

Alex Strickland ha scritto:
> D.Campagna wrote:
> 
>> Here I am with first results of testing.
>> Sorry for i'm late, I had some drawbacks unrelated with this argument.
> 
> I'm sorry I have not really followed the details of this but:
> 
>  > But now I'll try the real environment, where things are going wrong. 
> Maybe
>  > the antivirus? They have Panda and McAfee, while I have Avast or 
> nothing at
>  > all.
> 
> I have found McAfee to be a real dog about opening files, if you can 
> switch both of these for a short test I would do it.
> 
> Regards
> Alex
Thanks Alex, McAfee after having been IMO quite a good product in the 
past years, has definitely annoyed me and I am dumping it in favor of 
Panda and Kaspersky. Not only it is heavy, but the last version has 
problems in recognizing the licence number provided (a new idea) and 
forcing the user (or the security referrer, me) to uninstall completely 
with MCRT (or whatever - the McAfee Removal Tool), download the right 
version, reinstall etc. A pain. All this time-consuming task costs a lot 
more than the product itself and it's hard to make understand it to 
customers.
Anyway, the main issue was due to the deleted records at the beginning 
of a quite large file (10000 deleted over a total of 60000). I still 
don't get why DBU opened the file in half a second, while my application 
took 9/15 seconds depending on the PC involved, but right now, after a 
pack, things are OK.
I want to continue investigating if it's possible to improve 
performances, but now the thing is less compelling... :-)
It seems that my laptop with Avast, a Celeron with 384 Mb running XP 
Home, not really very powerful, is a rocket in comparison to a desktop 
Athlon 1800+ with Panda, 512 Mb and XP home on the same network.
But I'm not sure I can blame Panda, more testing is needed.
Regards

Dan

dlzc ha scritto:
> Dear D.Campagna:
> 

> 
> Surely not an argument...
And forget my lazy wording...
"argomento" is probably better translated with "subject" :-)

> 
>>> Can you give us an update on this?
>>> Do you access the remote resources via
>>> "drive:/path/filename" or "\\servername\path\filename"?
>> Good point, but I kept things very dumb. Since the
>> application resides in the same directory of data,
>> it accesses the tables via their name only, without
>> any path.
> 
> Not good enough.  Can you display / record drive and path information
> for the files from inside your application?
> 
Well, I simply pass tho the open DBF function the filename:
fhandle=sga("myfile") and other parameters. Then sga tries a net_use, 
quite the same function in the example file 'Locks.prg' from good old 
Clipper if shared access is requested, and returns 0 if operation 
failed. I map usually the remote shares with DOS-style letters, e.g. 
k:\whatever\myfile...
The application is started by a batch file that tests the existence of 
k:\whatever\myapp.exe and if not tries e.g.
net use k: \\server\public. Very basic.

> 
> ....
> I'd suggest you parse your SGA function, and record times for each
> step within it.

I think I should really do. I don't get why DBU did not show any lag, 
opening the infamous file in half a second.

>> A note must be given: the old Win98 machine, with
>> an obsolete hardware and a network card only able
>> to 10 Mb/sec (but it is the legendary 3Com
>> double-buffering network card...) acts as new powerful
>> machines. Gosh!
> 
> Yes, "advancement" as the marketers want to tell us is surely NOT
> improvement.  But then, I tend to buy based on "value" rather than
> absolute specs...

Still, not an easy task...

> 
> I use ClamWin.  AVG had to die because it interfered with gaming.

Really? I gave a try to Clamwin but at the time it hadn't real-time 
monitoring (and IIRC real-time mail scan)... but it was some years ago.
AVG performs quite well in my experience.

> You are making this more complex, clouding the issue in my opinion.
> Clipper apps work via NTVDM, and do so via an assigned drive letter
> for a network resource.  I suspect your DBU variant in xHarbour *also*
> access the files via a drive letter assignment (unless you seriously
> modified its code).
It's the old DBU, I only added REQUEST DBFCDX and RDDSETDEFAULT "DBFCDX".

> 
> Map a drive letter to the remote resource, and try accessing the files
> that way with xHarbour.  Just for grins.  Please.  This is a known and
> documented slowdown for file access in Windoze.

I was not aware of it. Many thanks. All my applications, as I said, work 
that way. Oops. It seems I have to review my code.
> 
> David A. Smith

Dan

Dear D.Campagna:

We've about beat this horse until it is fully dead...

On Jan 19, 10:37=A0am, "D.Campagna" <retsamt...@danielecampagna.co.cc>
wrote:
> dlzc ha scritto:
....
> > Surely not an argument...
>
> And forget my lazy wording...
> "argomento" is probably better translated
> with "subject" :-)

Or topic... and "subject" would be more common usage.

> > ....
> > I'd suggest you parse your SGA function,
> > and record times for each step within it.
>
> I think I should really do. I don't get why DBU
> did not show any lag, opening the infamous
> file in half a second.

Worth spending an hour on, if you have it.  *It* does not use "SGA()".

....
> > I use ClamWin. =A0AVG had to die because it
> > interfered with gaming.
>
> Really?
> AVG performs quite well in my experience.

It worked well until I invoked "Halflife" or something similar, then
the games locked up.  May have since been fixed.  Your customers may
not have the "attention span" issues I do... ;>)

> I gave a try to Clamwin but at the time it hadn't
> real-time monitoring (and IIRC real-time mail
> scan)... but it was some years ago.

Better now.  May still not be to your tastes.  Still free.  And I do
not insist everyone use the same virus scanner, since variety makes
the virus creaters job harder.

....
> > Map a drive letter to the remote resource,
> > and try accessing the files that way with
> > xHarbour. =A0Just for grins. =A0Please. =A0This is
> > a known and documented slowdown for
> > file access in Windoze.
>
> I was not aware of it. Many thanks. All my
> applications, as I said, work that way. Oops.
> It seems I have to review my code.

It will provide a penalty only on file open, so it seemd like a
candidate...  Try it once, and if it is noticeable, *then* revist your
code.  Otherwise, it is "ancient lore" and not worth worrying about
anymore.

No response required.

David A. Smith

>
> I think I should really do. I don't get why DBU did not show any lag, 
> opening the infamous file in half a second.

Perhaps a different setting of SET DELETED ON / OFF
Not familiar with DBU as to whether you can set this option before opening a 
file.

Harold

"dlzc" <dlzc1@cox.net> ha scritto nel messaggio 
news:17341e88-faed-4bc1-946b-90dd88b57aef@b2g2000yqi.googlegroups.com...

> > AVG performs quite well in my experience.
>
> It worked well until I invoked "Halflife" or something similar, then
> the games locked up.

I use AVG since many years. Honestly, I don't like it much due to the 
slowness of its shield component. Anyway, I'm a gamer and played many 3D 
games in the years without having any problems with AVG.

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

"H Davis" <hbdavis@sympatico.ca> ha scritto nel messaggio 
news:hj4vri$2lo$1@news.eternal-september.org...

> Perhaps a different setting of SET DELETED ON / OFF

Yes, I think so too.

EMG

--
EMAG Software Homepage:     http://www.emagsoftware.it
The EMG's ZX-Spectrum Page: http://www.emagsoftware.it/spectrum
The Best of Spectrum Games: http://www.emagsoftware.it/tbosg
The EMG Music page:         http://www.emagsoftware.it/emgmusic 

David,

>What is the status of "opportunistic locking"?  (Note that this should
>have no effect on opening a file, but on moving about in it.)

Actually, it does. When UseOpportunisticLocking is turned on (set to
1) on the workstations and EnableOplocks is turned on (set to 1) on
the server, an oplock on the entire file will be granted the first
workstation that opens the file in non-exclusive mode. When an
additional workstation requests opening of the same file, the server
must negotiate the oplock with the first workstation, and the first
workstation must flush writes and locks and release the oplock. This
may take some time, and the second workstation will have to wait for
it to finish.

See Q129202, for example

Regards,
Klas

-------
klas dot engwall at engwall dot com

http://www.engwall.com/clipper/

The LFN Library for Clipper
The LanMan Library for Clipper
The NFPAT1A Timeslice release patch for the Nanforum Toolkit

Dear Klas Engwall:

On Jan 19, 4:02=A0pm, klas.engw...@nospam.please (Klas Engwall) wrote:
<dlzc incorrectly said>
> > What is the status of "opportunistic locking"? =A0(Note that
> > this should have no effect on opening a file, but on moving
> > about in it.)
>
> Actually, it does. When UseOpportunisticLocking is turned
> on (set to 1) on the workstations and EnableOplocks is
> turned on (set to 1) on the server, an oplock on the entire
> file will be granted the first workstation that opens the file
> in non-exclusive mode. When an additional workstation
> requests opening of the same file, the server must negotiate
> the oplock with the first workstation, and the first
> workstation must flush writes and locks and release the
> oplock. This may take some time, and the second workstation
> will have to wait for it to finish.
>
> See Q129202, for example

Thanks for the correction.

If the server is Linux, as in this case (the OP has a Linux server,
some Windoze machines that have issues on opening, and internet users
that are happy as clams)... there may be no effect?

David A. Smith

D.Campagna wrote:

> Anyway, the main issue was due to the deleted records at the beginning
> of a quite large file (10000 deleted over a total of 60000). I still
> don't get why DBU opened the file in half a second, while my application
> took 9/15 seconds depending on the PC involved, but right now, after a
> pack, things are OK.
> I want to continue investigating if it's possible to improve
> performances, but now the thing is less compelling... :-)

Indexes?

Glad you at least got the thing sorted.

Regards
Alex

David,

>Thanks for the correction.
>
>If the server is Linux, as in this case (the OP has a Linux server,
>some Windoze machines that have issues on opening, and internet users
>that are happy as clams)... there may be no effect?

I suppose that depends on how closely the Samba folks have followed
Microsoft's footsteps regarding the stupid parts of the MS networking
default settings <g>. Sorry, I don't have the answer to that question.

But what is quite interesting in Dan's case is that although he had a
classic Clipper problem (tons of deleted records) that caused massive
delays for the SMB users, the same files could, if I understood it
correctly, be opened instantly by the CGI. Could it be that the CGI
does the same skip over all those deleted records so much more
efficiently than the SMB clients do so the real problem just isn't
noticed?

Regards,
Klas

-------
klas dot engwall at engwall dot com

http://www.engwall.com/clipper/

The LFN Library for Clipper
The LanMan Library for Clipper
The NFPAT1A Timeslice release patch for the Nanforum Toolkit

D.Campagna ha scritto:
Many thanks to all, and a special thank to H. Davis that put me in the 
right track.
The problem has been solved, it was quite a trivial one, the well-known 
delay in opening a file with SET DELETED ON  and a lot of deleted record 
at the beginning (for 10000 record, the delay was 9-15 seconds).

What misleaded me was the fast behavior of the CGI. So I implicitly 
excluded issues due to the language/data structure itself. True, it runs 
on Linux, but it's the same (x)Harbour for both programs. And SET 
DELETED ON is executed by the CGI, too.
As Klas Engwall pointed out with sublety, this difference could show 
that the SMB protocol is not really so efficient.
But we must consider that The CGI has local access to the files, and 
they reside on a native Linux filesystem, so the access is way more 
efficient.

Anyway It seems that others have more serious problems about slow .dbf 
access.
It could be interesting to follow the evolution of those cases.
Dan

Hello,

>> So now I'll try to connect an 'ancient' client running w98 and execute the
>> program from it. If it's faster, it's definitely a client problem.
>> Stay tuned if you are interested in the testing!
> 
> I'm really interesting!

Same here.

My customers can't even use my application with Windows Small Business 
Server. Just WAY to slow.. :(

-- 
Patrick

Patrick Mast ha scritto:
> Hello,
> 
>>> So now I'll try to connect an 'ancient' client running w98 and 
>>> execute the
>>> program from it. If it's faster, it's definitely a client problem.
>>> Stay tuned if you are interested in the testing!
>>
>> I'm really interesting!
> 
> Same here.
> 
> My customers can't even use my application with Windows Small Business 
> Server. Just WAY to slow.. :(
> 
ahem... it seems I have some stupid problem in connecting my old win98 
box to the samba server (it can't authenticate), so I reconfigured the 
network to use another PC with Win XP.

I posted the results, IIRC. Seeking...
Found!
 From my previous post:

Environment:
Server Linux (Zenwalk), kernel 2.6.25.4, Samba is 3.2.3
A small standard network with a couple of switch 100 Mb/sec
PC 1, Celeron 1.4 Ghz with XP Home SP3, 384 Mb RAM - runs Avast 
antivirus (if it matters)
PC 2, Athlon 64 3500+, 2Gb RAM, XP Professional SP2 (not updated) - runs 
Avast too and a bunch of resident programs (Daemon tools, Messenger 
etc.) To test Wine it runs Slackware 12.
PC 3, an old K6/500 running Windows 98 (Yes, I know... no comment 
please...) no antivirus at all.
PC 4: a Duron 1.1 Ghz with 512 Mb RAM, running Linux Zenwalk or Windows 
XP Professional SP2, always as a server.
 >
 > What is the status of "opportunistic locking"?  (Note that this should
 > have no effect on opening a file, but on moving about in it.)
 >
 > David A. Smith
I didn't (still) touch any parameter in the win registry, nor set any 
dbflockscheme. All plain vanilla.

Method of testing:
I did not put up a test program, rather added the following code to my 
existing application:
// put up a flag
log_op:=.t.

Added the following before and after file operations:
if log_op
    reglog(<cmessage>)
endif

function reglog is:

function reglog(ctxt)
local curf:=select()
public hlf:=0
if .not. file ("logfile.dbf")

dbcreate("logfile",{{"data","D",10,0},{"ora","C",8,0},{"user","N",2,0},{"oper","C",60,0},{"secondi","N",8,2},{"diff","N",8,2}})
endif
if hlf=0
    hlf=sga("logfile") // means more or less dbusearea, returns area 
number (select())
else
    select(hlf)
    if lower(alias())!="logfile"
      hlf=sga("logfile")
      reglog("Reopen logfile")
    endif
endif
pvsec=secondi
cursec=seconds()
append blank
replace data with date()
replace ora with time()
replace user with nodeid
replace oper with ctxt
replace secondi with cursec
replace diff with cursec-pvsec
select(curf)
return nil

This function creates "logfile" and keeps it opened, then adds a record 
with the timings.
The application is compiled with xHarbour latest cvs and WVW. RDD is 
CDX. All files are opened in shared mode.
Files are  varying in size from very small to 2 Mb. Real application 
(test will follow as soon as I'll put my hands on the REAL environment) 
has files about 40 Mb wide. And the problems.

Test 1
------
only one pc accesses data (pc 1) win->server Linux
open a small file 0,02
open a 165K file 4,19
open a small file 0,01
open file 1,2 Mb + 2 indexes 64 Kb and 25 Kb: 0,04
opens 2 more files quite small: 0,06
open file 2 Mb, (with 386K .FPT) + 5 indexes:  0,06
Set relation to some files: 0,01/not measured (0,00)
Some other file opened: 0,11

The second PC does the same operations, while files are in use by the 
first one  (pc 2) win->server Linux
open a small file 0,01
open a 165K file 2,57
open a small file 0,02
open file 1,2 Mb + 2 indexes 64 Kb and 25 Kb: 0,06
opens 2 more files quite small: 0,08
open file 2 Mb, (with 386K .FPT) + 5 indexes:  0,11
Set relation to some file: 0,02
Some other files opened: 0,14

Note to test 1
--------------
Reverting the order, say the Celeron-powered enters as second, shows 
that the more powerful the pc, the fastest the access. But we are 
talking of tenth of seconds.
I've also not been able to access from the Win98 PC the share, after a 
bit of tinkering I gave up. Dunno. Samba 3.xx maybe needs some 
configuration to accord with such an ancient thing.

Now let's change the server, PC 4 acts now as a server under Windows XP 
professional SP2 not upgraded.

Test 2
------
Pc 1 accesses data (WinXP->server WinXP)
open a small file 0,03
open a 165K file n.d.
open a small file 0,01
open file 1,2 Mb + 2 indexes 64 Kb and 25 Kb: 0,08
opens 2 more files quite small: 0,10
open file 2 Mb, (with 386K .FPT) + 5 indexes:  0,11
Set relation to some files: 0,01
Some other files opened: 0,14

(pc 2) winXP->server WinXP
open a small file 0,03
open a 165K file 4,57
open a small file 0,01
open file 1,2 Mb + 2 indexes 64 Kb and 25 Kb: 0,09
opens 2 more files quite small: 0,11
open file 2 Mb, (with 386K .FPT) + 5 indexes:  0,11
Set relation to some file: 0,02
Some other file opened: 0,19

Pc3 (note: with a 10 Mb/sec network card) Win98->server WinXP
open a small file 0,00 (not measurable)
open a 165K file 4,78
open a small file 0,00 (not measurable)
open file 1,2 Mb + 2 indexes 64 Kb and 25 Kb: 0,06
opens 2 more files quite small: 0,11
open file 2 Mb, (with 386K .FPT) + 5 indexes:  0,16
Set relation to some file: 0,00 (not measurable)
Some other file opened: 0,22

And so on.
Conclusions:
In my network, everything seems to work smoothly, at least for 3 users. 
I tried to start some other sessions from the XP clients with enough 
resources, and I did not experienced significant lags. A note must be 
given: the old Win98 machine, with an obsolete hardware and a network 
card only able to 10 Mb/sec (but it is the legendary 3Com 
double-buffering network card...) acts as new powerful machines. Gosh!

HTH
Dan