f



Delphi D4: using pointers to transfer data from COMM queue to circular queue

I have an application that uses a serial port component to read and write 
data through a USB serial port at 57.6 kb. Originally I used the SerialNG 
component and with some tweaking I was able to get it to work reliably. But 
it has problems with Win10, so I changed the code to use ComDrv32, which 
works when used in a simple TTY demo.

The SerialNG component uses clusters or packets which can be set to a range 
of values. I am communicating with a USB device set up as a CDC serial 
emulator using Microchip PIC18F2550. It is basically a data acquisition unit 
that samples at 2400 samples/sec with a 10 bit ADC. I am splitting the data 
into two 5 bit values in 8 bit data characters, and I'm using the remaining 
three to identify the data as a MSB or LSB  (using one evem/odd bit), and a 
two bit value to determine if the data has been received in proper sequence.

Here is the function that receives the data from the serial port component:

===============================================================
procedure TfmCommLib.CommPortDriver1ReceiveData(Sender: TObject;
  DataPtr: Pointer; DataSize: Cardinal);
var
          RecdPtr: PChar;
          CSize, NCSize : Integer;
          n: Integer;
begin
  RecdPtr := PChar(DataPtr);
  RxDcount := DataSize;
  if RxDcount > MaxRxDcount then
    MaxRxDcount := RxDcount;
  if CommBufferPtr + DataSize >= MAXCOMM then begin
     move( DataPtr^, CommBuffer[CommBufferPtr], MAXCOMM-CommBufferPtr );
     n := DataSize - (MAXCOMM-CommBufferPtr);
     RecdPtr := RecdPtr+n;
     DataPtr := RecdPtr;
     CommBufferPtr := 0;
     move( DataPtr^, CommBuffer[CommBufferPtr], n );
     end
  else begin
       move( DataPtr^, CommBuffer[CommBufferPtr], DataSize );
       CommBufferPtr := CommBufferPtr + DataSize; end;
  CommCharCount := CommCharCount + DataSize;
  if(CommCharCount > MAXCOMM) then
       CommCharCount := MAXCOMM+1;
  end;
========================================================

It works, but I get errors after running for about 20 seconds. That 
corresponds to the point where the CommBuffer will overflow the MAXCOMM 
value of 100,000 (4800*20). My previous code for SerialNG used a cluster 
size of 2 and a loop to transfer the received data to the buffer one byte at 
a time. When I tried that, it seemed to work poorly. So I used a variation 
of the method from the ComDrv32 TTY demo using move( DataPtr^, 
CommBuffer[CommBufferPtr], DataSize ); But you can see that I had to split 
the transfer when near the end of the queue.

I had to use a separate PChar pointer RecdPtr, and I also had to use an 
integer for the pointer arithmetic. Apparently Delphi does not allow RecdPtr 
:= RecdPtr + (DataSize - (MAXCOMM-CommBufferPtr) ); RecdPtr + n is OK. I'm 
not sure if I could just advance the DataPtr in the same way, since it is 
declared as a generic Pointer type.

I can't very well use the Delphi debugger to see what's happening, as the 
data is received continuously and there is no handshaking.

TIA for any suggestions.

Paul 

0
P
8/19/2016 6:19:06 AM
comp.lang.pascal.delphi.misc 5769 articles. 1 followers. miniFAQ (1) is leader. Post Follow

10 Replies
126 Views

Similar Articles

[PageSpeed] 6

I mentioned elsewhere that I used TCriticalSections previously during the 
transfers from the ComDrv32 component input buffer to the larger CommBuffer, 
and also when processing the CommBuffer to detect and compensate for errors. 
This seemed to work for the SerialNG component, but for ComDrv32 it seemed 
to increase the errors. Removing them, I got some errors starting at 20 
seconds as before, but after 700 seconds there are only 146 errors compared 
to nearly 1000 previously. There is probably a thread running to receive the 
characters into the COM16 file.

I tried various settings for the serial port component with inconsistent 
results. What is consistent is that errors always start when the CommBuffer 
is nearly full and the data must be split to fill the buffer and then start 
again at the beginning. I may need to check my arithmetic.

Paul 

0
P
8/19/2016 7:58:01 AM
I was able to rewrite the receive procedure and now it works with no errors. 
I think I have used pointers properly here:

==========================================================
procedure TfmCommLib.CommPortDriver1ReceiveData(Sender: TObject;
  DataPtr: Pointer; DataSize: Cardinal);

var
   RecdPtr: PChar;
   n: Integer;
begin
   RecdPtr := PChar(DataPtr);
   RxDcount := DataSize;
   if RxDcount > MaxRxDcount then
      MaxRxDcount := RxDcount;
   For n:= 1 to RxDcount do begin //2
     CommBuffer[CommBufferPtr] := RecdPtr^;
     inc(RecdPtr);
     inc(CommBufferPtr);
     if(CommBufferPtr > MAXCOMM) then
       CommBufferPtr := 0;
     inc(CommCharCount);
     if(CommCharCount > MAXCOMM) then
       fmCommLib.CommCharCount := MAXCOMM+1;
     end; //2
   end;
========================================================

In the previous procedure for SerialNG, I did this:

var
    RecdPtr: PChar;
begin
    RecdPtr := CommSerialPort.ReadNextCluster( NCSize, RdErrors );
    ...
    FreeMem( RecdPtr, NCSize );         // Added 4/8/09
end;
========================================================

I don't know if that was a problem or not. It seems that I might need to 
reserve memory for the PChar before using it?

Paul 

0
P
8/19/2016 8:42:54 AM
P E Schoen schrieb:
> I have an application that uses a serial port component to read and 
> write data through a USB serial port at 57.6 kb. Originally I used the 
> SerialNG component and with some tweaking I was able to get it to work 
> reliably. But it has problems with Win10, so I changed the code to use 
> ComDrv32, which works when used in a simple TTY demo.

I don't remember what prevented programs from using the asynchronous I/O 
API. 57kbd also doesn't look like a dangerous transmission speed.


> I had to use a separate PChar pointer RecdPtr, and I also had to use an 
> integer for the pointer arithmetic. Apparently Delphi does not allow 
> RecdPtr := RecdPtr + (DataSize - (MAXCOMM-CommBufferPtr) ); RecdPtr + n 
> is OK. I'm not sure if I could just advance the DataPtr in the same way, 
> since it is declared as a generic Pointer type.

Delphi allows pointer math only with PCHAR. At least I'd use Cardinal 
instead of Integer, in a workaround.

DoDi
0
Hans
8/19/2016 10:27:25 AM
P E Schoen schrieb:

> I tried various settings for the serial port component with inconsistent 
> results. What is consistent is that errors always start when the 
> CommBuffer is nearly full and the data must be split to fill the buffer 
> and then start again at the beginning. I may need to check my arithmetic.

Can't you use multiple buffers, to eliminate the moving of data?

DoDi
0
Hans
8/19/2016 10:30:04 AM
P E Schoen schrieb:

> In the previous procedure for SerialNG, I did this:
> 
> var
>    RecdPtr: PChar;
> begin
>    RecdPtr := CommSerialPort.ReadNextCluster( NCSize, RdErrors );
>    ...
>    FreeMem( RecdPtr, NCSize );         // Added 4/8/09
> end;
> ========================================================
> 
> I don't know if that was a problem or not. It seems that I might need to 
> reserve memory for the PChar before using it?

Various implementations of ReadNextCluster are possible. It may be 
required or forbidden to free the memory of the returned pointer.

DoDi
0
Hans
8/19/2016 10:34:55 AM
"Hans-Peter Diettrich"  wrote in message 
news:e1o5msFhoooU2@mid.individual.net...

> P E Schoen schrieb:

>> I tried various settings for the serial port component with inconsistent 
>> results. What is consistent is that errors always start when the 
>> CommBuffer is nearly full and the data must be split to fill the buffer 
>> and then start again at the beginning. I may need to check my arithmetic.

> Can't you use multiple buffers, to eliminate the moving of data?

Much of this code dates back to around 1996-1999 when I first tried making 
an application for Windows using Borland C++, and my first use of Borland 
Delphi 4 seems to be around 2003, when a consultant for another project 
showed me how to use it. I think I purchased Delphi and BC++ around 1998.

I'm not sure how multiple buffers would help. The serial port component has 
its own inBuffer which I have set to 48,000 bytes, or enough for 10 seconds 
of data. There is probably another small buffer at a lower level (there is 
an FTempInBuffer which is a pointer to data space allotted to 
FInBufferSize). The actual reading of data is accomplished by using:

nRead := 0;
nToRead := comStat.cbInQue;
if (nToRead > 0) and ReadFile( FHandle, FTempInBuffer^, nToRead, nRead, 
nil ) then
  if (nRead <> 0) and Assigned(FOnReceiveData) then
    FOnReceiveData( Self, FTempInBuffer, nRead );

So it does not appear that the FTempInBuffer is a circular queue, and thus I 
need to implement that in my own code using my CommBuffer[]. I could 
probably use much smaller values for the buffer sizes, but that's not really 
an issue these days when most computers have at least 2g of memory. The 
RxDataCount displayed in my status dialog is usually 280-390 bytes, which 
may depend on the PollingDelay which I have set at 50 mSec. At 4800 char/sec 
that is 240 characters. The extra characters likely occur during the buffer 
processing procedures.

I am doing additional processing in a procedure called by a timer set at 
50-250 mSec. So that may account for more delays and accumulation of data in 
the buffers. My MaxRxdCount after 10 minutes is now 1758. I have found that 
other Windows processes can increase this number, but so far it does not 
seem to lose data.

Quite a lot of real time processing occurs during these update intervals. 
Character pairs are read from the CommBuffer circular queue, and are parsed 
to verify their validity according to the extra two bits in each character, 
while the remaining 6 bits in each are concatenated to form a 12 bit 
integer. The ADC is only 10 bits but I designed the system to use 12 bits if 
needed. The value is shifted by 2048 to get a signed integer.

The absolute value is compared to a threshold value and a number of samples 
to determine if the value (of current) is enough to indicate that a test is 
in progress. If so, a small circular queue is used for pre-trigger data and 
added to a Waveform data buffer (like a digital storage scope). When the 
value drops below threshold for a number of samples, the actual start and 
stop points of the waveform are calculated, and a true RMS computation is 
performed to give the value of current in the pulse. This may be repeated 
for up to five pulses of 60 Hz current in a single test, with off times of 
as long as 20 seconds (although usually 1 or 2).

Meanwhile, the true RMS value of the samples in each update interval is 
computed and displayed on the screen, giving a continuous RMS value at all 
times. The sampling intervals of 50, 100, 150, 200, and 250 mSec are chosen 
to use an integral number of zero crossings for either 50 or 60 Hz. At 2400 
samples per second, 50 mSec is 120 samples, and corresponds to 3 cycles at 
60 Hz or 2.5 cycles at 50 Hz. This measuring technique is used in DMMs to 
cancel the effects of line voltage noise. It also produces consistent true 
RMS readings when the start and end of the sampling period are not 
synchronized to the zero crossings.

You may have already "left the building" at this point, unless you find this 
sort of discussion interesting. I have been involved in the measurement of 
power line frequency current and voltage signals for a LONG time, close to 
40 years. Technology has changed since then, from mostly analog methods, 
then ADCs with early microprocessors like the 8085 and Z80, then signal 
processing boards with MSDOS, and now PICs and DSPs with Windows 10 or other 
OS.

Thanks for your valued suggestions and time.

Paul 

0
P
8/20/2016 12:54:54 AM
P E Schoen schrieb:

> You may have already "left the building" at this point, unless you find 
> this sort of discussion interesting. I have been involved in the 
> measurement of power line frequency current and voltage signals for a 
> LONG time, close to 40 years. Technology has changed since then, from 
> mostly analog methods, then ADCs with early microprocessors like the 
> 8085 and Z80, then signal processing boards with MSDOS, and now PICs and 
> DSPs with Windows 10 or other OS.

Currently I see similar (but hobbyist) projects in the Arduino forum. I 
never tried real time programming on Win32/64, nor did I use COM port 
components with Delphi. In the early PC days I did some data acquisition 
using GW-Basic, later with BCB on Win3. I'd no more try to make my hands 
dirty with real time programs for Windows, and in detail not with 
Delphi, be some old reliable (up to D7) or uncomfortable and bloated new 
(RAD Studio) versions. It's interesting to hear that you still are in 
business :-)

DoDi
0
Hans
8/20/2016 10:43:43 AM
"Hans-Peter Diettrich"  wrote in message 
news:e1qqjiF6cpqU3@mid.individual.net...

> P E Schoen schrieb:

>> You may have already "left the building" at this point, unless you find 
>> this sort of discussion interesting. I have been involved in the 
>> measurement of power line frequency current and voltage signals for a 
>> LONG time, close to 40 years. Technology has changed since then, from 
>> mostly analog methods, then ADCs with early microprocessors like the 
>> 8085 and Z80, then signal processing boards with MSDOS, and now PICs and 
>> DSPs with Windows 10 or other OS.

> Currently I see similar (but hobbyist) projects in the Arduino forum. I 
> never tried real time programming on Win32/64, nor did I use COM port 
> components with Delphi. In the early PC days I did some data acquisition 
> using GW-Basic, later with BCB on Win3. I'd no more try to make my hands 
> dirty with real time programs for Windows, and in detail not with Delphi, 
> be some old reliable (up to D7) or uncomfortable and bloated new (RAD 
> Studio) versions. It's interesting to hear that you still are in business 
> :-)

Yes, I'm still in business, although at age 67 I've been semi-retired for 
about 2 years. I still support my Ortmaster products, including the original 
ORTM-1 and ORTM-2 that use MSDOS and the LPT parallel port to interface to 
the data acquisition device I designed around 1994 and produced until about 
2004 when I designed the first ORTM-3 (serial interface) and later the 
ORTM-4 (USB interface). Some customers still have and use the ORTM-2 but it 
is becoming more difficult for them to find working computers with MSDOS (or 
Win95) and native parallel ports. That's good for me, because I offer the 
only viable upgrade option for modern Windows machines.

I have considered upgrading to a newer version of Delphi. I have heard that 
the even numbered releases (D4 and D6) were buggy and their odd-numbered 
successors (D5 and D7) were much better. I found some Delphi Enterprise 8 
products on eBay for under $200 and CodeGear Delphi 2007 Enterprise for 
$430. Embarcadero does not offer an upgrade deal from D4, and their lowest 
level (Pro) version that offers database support is $1405.
https://www.embarcadero.com/app-development-tools-store/delphi

Perhaps even better is the open source Free Pascal and the Lazarus IDE that 
has a look and feel very similar to the Borland Delphi IDE that I am used to 
with D4-Pro.
http://wiki.freepascal.org/Lazarus_Documentation
https://www.youtube.com/playlist?list=PLB24C56953A79987A

Thanks for the information and discussion.

Paul 

0
P
8/20/2016 11:15:48 PM
P E Schoen schrieb:

> I have considered upgrading to a newer version of Delphi. I have heard 
> that the even numbered releases (D4 and D6) were buggy and their 
> odd-numbered successors (D5 and D7) were much better.

That's correct for the Borland products, up to D7. Afterwards bad goals 
had been chosen, e.g. a competitor for Microsoft .NET, which failed 
miserably. Then the adaptation to the MS (WinAPI) help files took 
several years, and online help still is at <50% compared to D4 or D7. 
Next came the introduction of Unicode support and Firemonkey, and way 
too late a 64 bit compiler.

> I found some 
> Delphi Enterprise 8 products on eBay for under $200 and CodeGear Delphi 
> 2007 Enterprise for $430. Embarcadero does not offer an upgrade deal 
> from D4, and their lowest level (Pro) version that offers database 
> support is $1405.
> https://www.embarcadero.com/app-development-tools-store/delphi

Upgrading existing projects seems to be quite easy, at least I could run 
a D5 program very soon on XE and XE4, with only a few missed detail 
adaptations found later. The prices, well, I think they are due to a 
shrinking user base. New projects mostly use VS, but that's not an 
option for your existing Delphi projects.

> Perhaps even better is the open source Free Pascal and the Lazarus IDE 
> that has a look and feel very similar to the Borland Delphi IDE that I 
> am used to with D4-Pro.
> http://wiki.freepascal.org/Lazarus_Documentation
> https://www.youtube.com/playlist?list=PLB24C56953A79987A

I discontinued watching FPC and Lazarus development. Some Delphi 
projects may port easily, others can require much work.

DoDi
0
Hans
8/21/2016 10:50:38 AM
"Hans-Peter Diettrich"  wrote in message 
news:e1thpaFq4gbU1@mid.individual.net...

> P E Schoen schrieb:

>> Perhaps even better is the open source Free Pascal and the Lazarus IDE 
>> that has a look and feel very similar to the Borland Delphi IDE that I 
>> am used to with D4-Pro.
>> http://wiki.freepascal.org/Lazarus_Documentation
>> https://www.youtube.com/playlist?list=PLB24C56953A79987A

>> I discontinued watching FPC and Lazarus development. Some Delphi projects 
>> may port easily, others can require much work.

I installed them on my Win10 computer and I have had some success with 
simpler projects. I'm having trouble installing the ComDrv32 package, but I 
don't fully understand how they work in Delphi, although I was able to do it 
somehow. I was able to get around the problem by removing the component from 
the form and then manually creating the component.

Now that I think about it, I probably have two instances of the component in 
my project, which might cause problems.

Thanks,

Paul 

0
P
8/22/2016 11:12:46 AM
Reply: