Hi!

I'm trying to create a combined configuration tool and GUI for
Python/Bittorrent vio/cli (other downloaders and vio/cli apps should 
also be insteresting for other projects)

I'd like to get the output from the app to show the graph and progress +
other info in my GUI app and I've tried to run rxqueue and my own queue
but I've finally found that I should use pipes. Rxqueue won't release
the queue before the execution is done which results in that the GUI
can't show anything before the download is over. My own queue overcomes
that by sleeping a sec now and then, but it's to unreliable. Pipes and 
ipc should hopefully be more useful and open up possibilities to add GUI 
to many vio apps that's available and does a great job ( pkzip, 
bittorrent, wget ). Available solutions often assume that I have to 
write my own .cmd-wrapper first, something that i can't find resonable 
for a vio/cli app. :-( I'm just interested in the output from the 
vio/cli app. (to reformat it and present the info) and maybe killing it. 
I need to e.g. monitor 6 simultanous downloads for 19 hours in realtime 
with maintained responsiveness.

The documentation and examples for e.g. rxipc is a bit sparse and 
doesn't work the way I want and I've found no examples on the net.

Could you please help me? A little example would be most appreciated.

A working reliable solution would be nice...


Mvh / Wkr / MfG
//Jan-Erik

Jan-Erik wrote:
> 
> Hi!
> 
> I'm trying to create a combined configuration tool and GUI for
> Python/Bittorrent vio/cli (other downloaders and vio/cli apps should
> also be insteresting for other projects)
> 
> I'd like to get the output from the app to show the graph and progress +
> other info in my GUI app and I've tried to run rxqueue and my own queue
> but I've finally found that I should use pipes. Rxqueue won't release
> the queue before the execution is done which results in that the GUI
> can't show anything before the download is over. My own queue overcomes
> that by sleeping a sec now and then, but it's to unreliable. Pipes and
> ipc should hopefully be more useful and open up possibilities to add GUI
> to many vio apps that's available and does a great job ( pkzip,
> bittorrent, wget ). Available solutions often assume that I have to
> write my own .cmd-wrapper first, something that i can't find resonable
> for a vio/cli app. :-( I'm just interested in the output from the
> vio/cli app. (to reformat it and present the info) and maybe killing it.
> I need to e.g. monitor 6 simultanous downloads for 19 hours in realtime
> with maintained responsiveness.
> 
> The documentation and examples for e.g. rxipc is a bit sparse and
> doesn't work the way I want and I've found no examples on the net.
> 
> Could you please help me? A little example would be most appreciated.
> 
> A working reliable solution would be nice...
> 
> Mvh / Wkr / MfG
> //Jan-Erik

Hi,

This being OS/2/Ecs?
You could send the cli-output into a logfile and use logv[iew] in PM to monitor
the logs in realtime.
The Version on hobbes is 1.04, but I got 1.06 (dated 29.01.2002) somewhere.

IMSC there might be a program, which catches any output directly, but ...?

Wolfgang

May I suggest asking this question in comp.os.os2.programmer.misc?

 > Pipes should hopefully be more useful and open up possibilities
 
AFAIK then the CLI app already has to fill the (known-by-name) pipe,
so you can connect to that pipe, just like the output would we written
to a real-time logfile too. 

Not sure if it's possible at all, but you may end up with an app that
monitor other VIO windows (somehow), but forget true IPC. The other
side has to be IPC-aware, and it won't bleed if you stick a pipe in
it, which would be a nice virus-entry too, I assume.



---

Wolfgang Riedel wrote:

>Jan-Erik wrote:
>  
>
>>Hi!
>>
>>I'm trying to create a combined configuration tool and GUI for
>>Python/Bittorrent vio/cli (other downloaders and vio/cli apps should
>>also be insteresting for other projects)
>>
>>I'd like to get the output from the app to show the graph and progress +
>>other info in my GUI app and I've tried to run rxqueue and my own queue
>>but I've finally found that I should use pipes. Rxqueue won't release
>>the queue before the execution is done which results in that the GUI
>>can't show anything before the download is over. My own queue overcomes
>>that by sleeping a sec now and then, but it's to unreliable. Pipes and
>>ipc should hopefully be more useful and open up possibilities to add GUI
>>to many vio apps that's available and does a great job ( pkzip,
>>bittorrent, wget ). Available solutions often assume that I have to
>>write my own .cmd-wrapper first, something that i can't find resonable
>>for a vio/cli app. :-( I'm just interested in the output from the
>>vio/cli app. (to reformat it and present the info) and maybe killing it.
>>I need to e.g. monitor 6 simultanous downloads for 19 hours in realtime
>>with maintained responsiveness.
>>
>>The documentation and examples for e.g. rxipc is a bit sparse and
>>doesn't work the way I want and I've found no examples on the net.
>>
>>Could you please help me? A little example would be most appreciated.
>>
>>A working reliable solution would be nice...
>>
>>Mvh / Wkr / MfG
>>//Jan-Erik
>>    
>>
>
>Hi,
>
>This being OS/2/Ecs?
>You could send the cli-output into a logfile and use logv[iew] in PM to monitor
>the logs in realtime.
>The Version on hobbes is 1.04, but I got 1.06 (dated 29.01.2002) somewhere.
>
>IMSC there might be a program, which catches any output directly, but ...?
>
>Wolfgang
>  
>
Thanks, I'll try that!

I can do it if i create/use my own wrapper , one just have to queue 
linein(); in a loop and then your set. The problem is though that queues 
doesn't work in good in realtime. Queues doesn't involve reading/writing 
to/from files and it's possible to control other .cmd-files with queues.

If someone could tell me how to assign a pipe to an app when it's 
started on the commandline then this issue would probably be solved.

OS/2 and eCS can do a lot more than many other systems can, one just 
don't know it's full potential. Capturing output from a vio/cli app is 
very easy to do, but I don't know what to do next.

Mvh / Wkr / MfG
//Jan-Erik

Jean wrote:

>May I suggest asking this question in comp.os.os2.programmer.misc?
>
> > Pipes should hopefully be more useful and open up possibilities
> 
>AFAIK then the CLI app already has to fill the (known-by-name) pipe,
>so you can connect to that pipe, just like the output would we written
>to a real-time logfile too. 
>
>Not sure if it's possible at all, but you may end up with an app that
>monitor other VIO windows (somehow), but forget true IPC. The other
>side has to be IPC-aware, and it won't bleed if you stick a pipe in
>it, which would be a nice virus-entry too, I assume.
>
>
>
>---
>  
>
Hmm. Pipes and queues can have unique names and they should therefore 
only be monitored by the names, that's how I can keep them apart today, 
when I use queues.

Virus? I don't quite follow you there...

 If there would be a soultion that resembles queues I'd be most delighted.

e.g. something like...

1) Create Pipe
  newpipe = rxPipe( 'Create' ); /* Create a new unique pipe to 
attach/listen to */
2) Set the new pipe as active.
  IF rxPipe( 'Set', newpipe ) THEN DO /* Set it as active */
3) Execute vio/cli app. asyncronously (immediate return)
  rc = rxExec( 'DIR', '*.cmd /F /S', newpipe ); /* Execute app and bind 
it's output to pipe by name */
4) Loop and listen
  DO FOREVER      /* Listen for output forever */
    IF rxPipe( 'Query', newpipe ) THEN DO /* vio/cli app still running? */
      IF LINES() THEN DO                  /* Output from app? */
        data = LINEIN( newpipe );      /*  read it then */
        IF data <> '' THEN DO             /* not empty? */
          file = FILESPEC('N', data);      /* adjust the format to fit 
our needs! */
          IF TIME( 'E' ) > timer THEN DO  /* e.g. every 2 sec */
            CALL Notify 'draw_value', file;/* tell thread to draw */
            CALL TIME( 'R' );             /* reset timer */
          END
        END
        ELSE
          LEAVE;                          /* else leave */
      END
      ELSE
        LEAVE;                            /* else leave */
    END
    DO WHILE \LINES() | TIME( 'E' ) < timeout/* until lines present or 
e.g. 10 sec. */
      CALL SysSleep 1;                    /* wait 1 sec. at the time */
    END
  END
5) Close pipe and Ctrl + Break (Kill/stop execution of vio/cli app if 
needed)
rc = rxPipe( 'Close', newpipe );    /* Close the pipe from futher use */
6) Done!

This example is a downscaled version of the one I'd like to see. It 
resembles the version I use together with queues.

Mvh / Wkr / MfG
//Jan-Erik

On Fri, 28 Nov 2003 00:37:18 +0100, Jan-Erik hath writ:
> 
> If someone could tell me how to assign a pipe to an app when it's 
> started on the commandline then this issue would probably be solved.

I hope I understand you correctly.  I have some web (curl), VIO, 
REXX programs that scrape information from web pages on a 
once-every-nn-minutes schedule and pass that information through a 
pipe to several other REXX, VIO programs that (depending on the
application) cull, trim, and reformat the information, calculate 
some new data points, and display the "new" information in a "properly"
sized VIO window.

The display programs are started by the web program(s) using ADDRESS CMD 
The `unique`, selected pipe name is passed as a parameter on the 
ADDRESS CMD command line -- when the web program(s) _first_ start up.
Thereafter, the "web programs" merely pump data into the selected pipes.
The "VIO display" programs merely read data from their `unique` pipe.

Truly, the BIGGEST headache is the maintenance of the pipes.
How - when - to clean them up and delete them.
Since I'm at the mercy of ever changing web site (re)formats, I
get to visit this problem fairly often.

Is that what you mean?

HTH
Jonesy
-- 
  | Marvin L Jones       | jonz         |  W3DHJ   |  OS/2
  |  Gunnison, Colorado  |  @           |  Jonesy  |  linux   __
  |   7,703' -- 2,345m   |   config.com |  DM68mn             SK

Allodoxaphobia wrote:

>On Fri, 28 Nov 2003 00:37:18 +0100, Jan-Erik hath writ:
>  
>
>>If someone could tell me how to assign a pipe to an app when it's 
>>started on the commandline then this issue would probably be solved.
>>    
>>
>
>I hope I understand you correctly.  I have some web (curl), VIO, 
>REXX programs that scrape information from web pages on a 
>once-every-nn-minutes schedule and pass that information through a 
>pipe to several other REXX, VIO programs that (depending on the
>application) cull, trim, and reformat the information, calculate 
>some new data points, and display the "new" information in a "properly"
>sized VIO window.
>
>The display programs are started by the web program(s) using ADDRESS CMD 
>The `unique`, selected pipe name is passed as a parameter on the 
>ADDRESS CMD command line -- when the web program(s) _first_ start up.
>Thereafter, the "web programs" merely pump data into the selected pipes.
>The "VIO display" programs merely read data from their `unique` pipe.
>
>Truly, the BIGGEST headache is the maintenance of the pipes.
>How - when - to clean them up and delete them.
>Since I'm at the mercy of ever changing web site (re)formats, I
>get to visit this problem fairly often.
>
>Is that what you mean?
>
>HTH
>Jonesy
>  
>
Yes! Please tell me how to do it/use it/ your solution.

Mvh / Wkr / MfG
//Jan-Erik

On Fri, 28 Nov 2003 20:04:09 +0100, Jan-Erik hath writ:
> Allodoxaphobia wrote:
> 
>>On Fri, 28 Nov 2003 00:37:18 +0100, Jan-Erik hath writ:
>>
>>>If someone could tell me how to assign a pipe to an app when it's 
>>>started on the commandline then this issue would probably be solved.
>>
>>I hope I understand you correctly.  I have some web (curl), VIO, 
>>REXX programs that scrape information from web pages on a 
>>once-every-nn-minutes schedule and pass that information through a 
>>pipe to several other REXX, VIO programs that (depending on the
>>application) cull, trim, and reformat the information, calculate 
>>some new data points, and display the "new" information in a "properly"
>>sized VIO window.
>>
>>The display programs are started by the web program(s) using ADDRESS CMD 
>>The `unique`, selected pipe name is passed as a parameter on the 
>>ADDRESS CMD command line -- when the web program(s) _first_ start up.
>>Thereafter, the "web programs" merely pump data into the selected pipes.
>>The "VIO display" programs merely read data from their `unique` pipe.
>>
>>Truly, the BIGGEST headache is the maintenance of the pipes.
>>How - when - to clean them up and delete them.
>>Since I'm at the mercy of ever changing web site (re)formats, I
>>get to visit this problem fairly often.
>>
>>Is that what you mean?
 
> Yes! Please tell me how to do it/use it/ your solution.

I thought I did.  :-)

In fairness, I used the term and verb "pipe" in my other post, 
but I'm using (REXX) queues.

In the "master"/"controlling" program -- at initialization --
you issue (crudely re-editted from one of my programs):

 |qname = some_unique_queue_name_for_this_application
 |
 |rc2 = "not_set"
 |rc1 = RxQueue("DELETE",qname)
 |
 |If rc1 < 10 Then
 |rc2 = RxQueue("CREATE",qname)
 |Else
 | Do
 |  Say "PROBLEM!  RxQUEUE(DELETE,"||qname||") rc ==" rc1
 |  screw_up = 5
 |  Signal failure
 | End
 |
 |Address cmd 'start /c /f VIODISPLAY.CMD' qname parm2 parm3.....

The started VIO program takes the passed Queue name and opens it for 
data retrieval from the "master"/"controlling" program.  In normal
operation it "understands" the data that is being passed across.
There are "special" text streams passed in the Queue to inform the
display program on "special" actions to take -- mostly debugging
requests, but most often it's a
 | 
 |Queue "::EOJ::" 
 |
command.  That should be casual to the most obvious observer.  :-)

I use "^C" in conjunction with Signal On HALT in the 
"master"/"controlling" program to bring down the whole mess.
Crude, yes -- but, it's KISS.

My code is not bullet-proof.  But, rather, it is "Good Enough".
I've adopted this rationalization due to the fact that I 
frequently need to tear it all apart and put it back together
again everytime the web site(s) I'm scraping from change in
format or content.  

One application gather data (prices) for stocks and the DJIA 
and NASD numbers.  I use curl to scrape the stock numbers from 
one site. But, because the (current) best, all-round stocklist 
display for _my_ purposes does not also display the DJIA and/or 
NASD, the program uses curl also to pull a page from elsewhere 
that _does_ display the DJI and NASD.  Eventually one or both of
these pages will change again and I will have to re-jigger my
program(s) again.
(Actually, this _has_ happened, and I am in the process of 
 re-doing the program(s) to pull data from a different URL.
 (Yahoo can be exacerbating.))

Another application gathers postings from amateur radio propagation
loggers.  I'm most interested in the VHF listings, so I clean up,
trim, and - yes - killfile the postings from 
  http://dxworld.com/50prop.html    and
  http://dxworld.com/144prop.html   and, optionally, 1 or 2 others.
Here, the "master"/"controlling" program uses curl in a "round robin"
fashion to periodically visit all the URL's found in a control file -- 
and has started up a VIO display program for each instance of a URL.

I use a lot of "after-market"/"add-on" programs in this miky-maus
programming.  :-)  I cannot remember all of them -- but, I've
googled for most of them as I've seen a need for a "utility"/"function".
To list some, but not all:
  curl          - to fetch web pages.
  htm2txt       - to strip html from web pages
  RexxSem.dll   - to set window titles, for one thing.

As with *all* programming projects, each set of programs is 
90 percent complete.  The axiom being: 90 percent of the program
requires 10 percent of the time.  The last 10 percent of the
program requires 90 percent of the time.

HTH
Jonesy
-- 
  | Marvin L Jones       | jonz         |  W3DHJ   |  OS/2
  |  Gunnison, Colorado  |  @           |  Jonesy  |  linux   __
  |   7,703' -- 2,345m   |   config.com |  DM68mn             SK

Allodoxaphobia wrote:

>On Fri, 28 Nov 2003 20:04:09 +0100, Jan-Erik hath writ:
>  
>
>>Allodoxaphobia wrote:
>>
>>    
>>
>>>On Fri, 28 Nov 2003 00:37:18 +0100, Jan-Erik hath writ:
>>>
>>>      
>>>
>>>>If someone could tell me how to assign a pipe to an app when it's 
>>>>started on the commandline then this issue would probably be solved.
>>>>        
>>>>
>>>I hope I understand you correctly.  I have some web (curl), VIO, 
>>>REXX programs that scrape information from web pages on a 
>>>once-every-nn-minutes schedule and pass that information through a 
>>>pipe to several other REXX, VIO programs that (depending on the
>>>application) cull, trim, and reformat the information, calculate 
>>>some new data points, and display the "new" information in a "properly"
>>>sized VIO window.
>>>
>>>The display programs are started by the web program(s) using ADDRESS CMD 
>>>The `unique`, selected pipe name is passed as a parameter on the 
>>>ADDRESS CMD command line -- when the web program(s) _first_ start up.
>>>Thereafter, the "web programs" merely pump data into the selected pipes.
>>>The "VIO display" programs merely read data from their `unique` pipe.
>>>
>>>Truly, the BIGGEST headache is the maintenance of the pipes.
>>>How - when - to clean them up and delete them.
>>>Since I'm at the mercy of ever changing web site (re)formats, I
>>>get to visit this problem fairly often.
>>>
>>>Is that what you mean?
>>>      
>>>
> 
>  
>
>>Yes! Please tell me how to do it/use it/ your solution.
>>    
>>
>
>I thought I did.  :-)
>
>In fairness, I used the term and verb "pipe" in my other post, 
>but I'm using (REXX) queues.
>
>In the "master"/"controlling" program -- at initialization --
>you issue (crudely re-editted from one of my programs):
>
> |qname = some_unique_queue_name_for_this_application
> |
> |rc2 = "not_set"
> |rc1 = RxQueue("DELETE",qname)
> |
> |If rc1 < 10 Then
> |rc2 = RxQueue("CREATE",qname)
> |Else
> | Do
> |  Say "PROBLEM!  RxQUEUE(DELETE,"||qname||") rc ==" rc1
> |  screw_up = 5
> |  Signal failure
> | End
> |
> |Address cmd 'start /c /f VIODISPLAY.CMD' qname parm2 parm3.....
>
>The started VIO program takes the passed Queue name and opens it for 
>data retrieval from the "master"/"controlling" program.  In normal
>operation it "understands" the data that is being passed across.
>There are "special" text streams passed in the Queue to inform the
>display program on "special" actions to take -- mostly debugging
>requests, but most often it's a
> | 
> |Queue "::EOJ::" 
> |
>command.  That should be casual to the most obvious observer.  :-)
>
>I use "^C" in conjunction with Signal On HALT in the 
>"master"/"controlling" program to bring down the whole mess.
>Crude, yes -- but, it's KISS.
>
>My code is not bullet-proof.  But, rather, it is "Good Enough".
>I've adopted this rationalization due to the fact that I 
>frequently need to tear it all apart and put it back together
>again everytime the web site(s) I'm scraping from change in
>format or content.  
>
>One application gather data (prices) for stocks and the DJIA 
>and NASD numbers.  I use curl to scrape the stock numbers from 
>one site. But, because the (current) best, all-round stocklist 
>display for _my_ purposes does not also display the DJIA and/or 
>NASD, the program uses curl also to pull a page from elsewhere 
>that _does_ display the DJI and NASD.  Eventually one or both of
>these pages will change again and I will have to re-jigger my
>program(s) again.
>(Actually, this _has_ happened, and I am in the process of 
> re-doing the program(s) to pull data from a different URL.
> (Yahoo can be exacerbating.))
>
>Another application gathers postings from amateur radio propagation
>loggers.  I'm most interested in the VHF listings, so I clean up,
>trim, and - yes - killfile the postings from 
>  http://dxworld.com/50prop.html    and
>  http://dxworld.com/144prop.html   and, optionally, 1 or 2 others.
>Here, the "master"/"controlling" program uses curl in a "round robin"
>fashion to periodically visit all the URL's found in a control file -- 
>and has started up a VIO display program for each instance of a URL.
>
>I use a lot of "after-market"/"add-on" programs in this miky-maus
>programming.  :-)  I cannot remember all of them -- but, I've
>googled for most of them as I've seen a need for a "utility"/"function".
>To list some, but not all:
>  curl          - to fetch web pages.
>  htm2txt       - to strip html from web pages
>  RexxSem.dll   - to set window titles, for one thing.
>
>As with *all* programming projects, each set of programs is 
>90 percent complete.  The axiom being: 90 percent of the program
>requires 10 percent of the time.  The last 10 percent of the
>program requires 90 percent of the time.
>
>HTH
>Jonesy
>  
>

The thing is that you'll never notice then is that you won't get the 
results until after the app has run if you use RXQUEUE with an exe and 
then it's reliable. The solution you've mentioned here is very similar 
to the one I'm using at the moment, but the failure rate for me isn't 
good enough though.

I'd like to see a .dll for rexx as e.g.
http://w1.652.telia.com/~u65204275/Pipe.png

Anyone up to it? Please!

Mvh / Wkr / MfG
//Jan-Erik

Hello Jan-Erik,

Jan-Erik schrieb:
> I'd like to get the output from the app to show the graph and progress +
> other info in my GUI app and I've tried to run rxqueue and my own queue
> but I've finally found that I should use pipes. Rxqueue won't release
> the queue before the execution is done which results in that the GUI
> can't show anything before the download is over. My own queue overcomes

Did you ever consider using Regina? Regina has an external queue server
called rxstack which can be accessed simultaneously by every process.
Starting up the data collector with a rxqueue as a filter, you can
fetch the gotten data from every machine in the network or just the
local machine.
It is possible to wait for data with a configurable timeout and you
can access one queue by as many programs as you like without locking up
the others.
This isn't very fast in opposite to internal queues, but if you don't
have more than about 1MB per second to transfer on a slow machine you
should give it a try.

Cheers, Florian

Florian Gro�e-Coosmann wrote:

> Hello Jan-Erik,
>
> Jan-Erik schrieb:
>
>> I'd like to get the output from the app to show the graph and progress +
>> other info in my GUI app and I've tried to run rxqueue and my own queue
>> but I've finally found that I should use pipes. Rxqueue won't release
>> the queue before the execution is done which results in that the GUI
>> can't show anything before the download is over. My own queue overcomes
>
>
> Did you ever consider using Regina? Regina has an external queue server
> called rxstack which can be accessed simultaneously by every process.
> Starting up the data collector with a rxqueue as a filter, you can
> fetch the gotten data from every machine in the network or just the
> local machine.
> It is possible to wait for data with a configurable timeout and you
> can access one queue by as many programs as you like without locking up
> the others.
> This isn't very fast in opposite to internal queues, but if you don't
> have more than about 1MB per second to transfer on a slow machine you
> should give it a try.
>
> Cheers, Florian
>
Not really. I'm writing my GUI in OS/2 with DrDialog or similar, I'd
also want others to benefit from such a tool/addon and not require to
get more than a .dll along with my app. Thanks for the suggestion
though, I may try it out to see it's potential and howit's done.

Mvh / Wkr / MfG
//Jan-Erik

Allodoxaphobia entertained comp.lang.rexx with the following story:
>As with *all* programming projects, each set of programs is 
>90 percent complete.  The axiom being: 90 percent of the program
>requires 10 percent of the time.  The last 10 percent of the
>program requires 90 percent of the time.

The first 90% of the project takes 90% of the time.  The last 10%
takes the other 90% of the time.
-- 
---- Ian Collier : imc@comlab.ox.ac.uk : WWW page (including REXX section):
------ http://users.comlab.ox.ac.uk/ian.collier/imc.shtml

New to this group?  Answers to frequently-asked questions can be had from
http://rexx.hursley.ibm.com/rexx/ .