f



tcl-mmap: A POSIX mmap interface for Tcl. (Tcl package)

This is an announcement for a relatively new Tcl project: tcl-mmap

Tcl-mmap is a Tcl interface to the POSIX mmap* system call. It provides 
a Tcl package that allows Tcl scripts to: 1) Memory map files for 
improved access efficiency; 2) Share memory between related processes; 
3) Easily implement cyclic persistent log files.

* See the mmap(2) man page.

Platform: Linux/Unix
Home page: http://sourceforge.net/projects/tcl-mmap/
Man page: http://tcl-mmap.sourceforge.net/
Author: Alexandros Stergiakis
0
alsterg (13)
9/3/2008 9:48:52 AM
comp.lang.tcl 23428 articles. 2 followers. Post Follow

16 Replies
3099 Views

Similar Articles

[PageSpeed] 16

On Sep 3, 11:48=A0am, Alexandros Stergiakis <alst...@gmail.com> wrote:
> This is an announcement for a relatively new Tcl project: tcl-mmap
>
> Tcl-mmap is a Tcl interface to the POSIX mmap* system call. It provides
> a Tcl package that allows Tcl scripts to: 1) Memory map files for
> improved access efficiency; 2) Share memory between related processes;
> 3) Easily implement cyclic persistent log files.
>
> * See the mmap(2) man page.
>
Great to see this and the other packages you made.

Looking at the manpage it looks a bit misformatted before the usage
example.

Any specific reason to use GPL for this instead the usual Tcl/MIT/BSD
style license used?

Michael
0
schlenk (1615)
9/3/2008 10:01:26 AM
schlenk wrote:
> On Sep 3, 11:48 am, Alexandros Stergiakis <alst...@gmail.com> wrote:
>> This is an announcement for a relatively new Tcl project: tcl-mmap
>>
>> Tcl-mmap is a Tcl interface to the POSIX mmap* system call. It provides
>> a Tcl package that allows Tcl scripts to: 1) Memory map files for
>> improved access efficiency; 2) Share memory between related processes;
>> 3) Easily implement cyclic persistent log files.
>>
>> * See the mmap(2) man page.
>>
> Great to see this and the other packages you made.
> 
> Looking at the manpage it looks a bit misformatted before the usage
> example.
> 
> Any specific reason to use GPL for this instead the usual Tcl/MIT/BSD
> style license used?
> 
> Michael
Yes the documentation needs some improvement, but I hope it is readable 
enough as it is now. Contributions are welcome.

All the FOSS software I'm writing is always released under GPL for 
personal ideological reasons.

alsterg
0
alsterg (13)
9/3/2008 3:04:56 PM
Hi Alexandros,

On Sep 3, 11:48=A0am, Alexandros Stergiakis <alst...@gmail.com> wrote:
> This is an announcement for a relatively new Tcl project: tcl-mmap

Congrats !

The idea of a channel interface is very nice, especially on the write
side where the immutability of Tcl values makes it rather tricky to
modify something that's reflected in (possibly shared) values.
The reason I'm saying that is that I was considering the
implementation of something similar, but as a variant of the ByteArray
value type. For the read side, the idea works quite well, allowing
[string range] to access the mapped area randomly. But for the write
side, I was a bit stuck in the mutability nightmare:

      set x [binary mmap $ch $off $len rw]
      set y $x ;# <-- hey now it's shared !
      binary poke x $off $somedata

the problem here being to choose between 2 evils: (a) allow for
mutability of (mmapped byte array) values, with the usual problems of
mutability (which can be milder than with other types since there's no
iterator on byte arrays today), and (b) arranhge for the code above to
"fork" the fates of x and y at some point (with a potentially huge
malloc and copy) or forbid the sharing altogether...

So your solution is in all respects superior: symmetry, reuse of
existing paradigms, etc.
Kudos !

-Alex

PS: one single thing that the channel style doesn't allow, and the
mmapped-byte-array permits nicely, is the "big string search": [string
first $needle $mmapped_haystack]. But I admit being at a loss finding
real-life uses of this (the main application of mmap being rather
large binary things with blocks reached by offset). I have no other
choice than admitting defeat :-}
0
9/3/2008 5:14:38 PM
Alexandre Ferrieux <alexandre.ferrieux@gmail.com> wrote:
> The idea of a channel interface is very nice, especially on the write
> side where the immutability of Tcl values makes it rather tricky to
> modify something that's reflected in (possibly shared) values.

I've once thought about proposing a "vector" to Tcl, which would
be mostly like an array (for the syntax to use it), but would be
confined to a defined range of integral indices offering real
constant time access (unlike hashmaps) and perhaps even less
memory overhead.

Next step could be to allow to impose restrictions on the values,
to the benefit of even more efficient storage, e.g. by leaving out
Tcl_Objects and storing just ints or bytes (but at the cost of
no longer supporting traces on single elements).  Of course this
doesn't break EIAS, because every command has the right to refuse
arguments that cannot be converted into some particular domain.
And no new structure is *required* to support traces.

So far, it was just one of my "ideas without use",  but it looks
like an "mmap"-feature could benefit from such a structure, and it
would not even get near the problem of mutable values, since it
wouldn't be a "value" in the Tcl-sense.

> So your solution is in all respects superior: symmetry, reuse of
> existing paradigms, etc.

I wouldn't be surprised, if a vector could also behave like 
a stream for reading (it's just a disguised iteration*, after
all)  and also behave like a writing stream filling the vector
with data.

Just brainstorming...

*: with iteration I mean: startsearch, anymore, nextelement, donesearch
0
avl1 (2748)
9/3/2008 6:42:06 PM
On Sep 3, 2:42=A0pm, Andreas Leitgeb <a...@gamma.logic.tuwien.ac.at>
wrote:
[snip]
> I've once thought about proposing a "vector" to Tcl, which would
> be mostly like an array (for the syntax to use it), but would be
> confined to a defined range of integral indices offering real
> constant time access (unlike hashmaps) and perhaps even less
> memory overhead.
>
> Next step could be to allow to impose restrictions on the values,
> to the benefit of even more efficient storage, e.g. by leaving out
> Tcl_Objects and storing just ints or bytes (but at the cost of
> no longer supporting traces on single elements). =A0Of course this
> doesn't break EIAS, because every command has the right to refuse
> arguments that cannot be converted into some particular domain.
> And no new structure is *required* to support traces.
> [snip]

Tangential ... but with respect to cpu/memory efficiency exists Karl
Lehenbauer's very cool Speed Tables:
http://www.tcl.tk/community/tcl2007/proceedings/databases/speedtables-paper=
-2.pdf


0
9/4/2008 3:06:29 AM
Alexandros Stergiakis wrote:
> This is an announcement for a relatively new Tcl project: tcl-mmap

Hey! you've been busy to no end. Thanks.

Had a look at dbus too ;-?

uwe

0
9/5/2008 10:46:42 AM
Uwe Klein wrote:
> Had a look at dbus too ;-?
> 
Are you aware there's dbus-tcl? http://dbus-tcl.sourceforge.net/

Schelte.
-- 
set Reply-To [string map {nospam schelte} $header(From)]
0
nospam6520 (256)
9/5/2008 6:03:08 PM
Schelte Bron wrote:
> Uwe Klein wrote:
> 
>>Had a look at dbus too ;-?
>>
> 
> Are you aware there's dbus-tcl? http://dbus-tcl.sourceforge.net/
> 
> Schelte.
err. NO.. Not until now, thanks.

uwe
0
9/5/2008 8:37:13 PM
Schelte Bron wrote:
> Uwe Klein wrote:
> 
>>Had a look at dbus too ;-?
>>
> 
> Are you aware there's dbus-tcl? http://dbus-tcl.sourceforge.net/
> 
> Schelte.
this requires tcl 8.5* ?

uwe
0
9/5/2008 8:59:05 PM
Uwe Klein wrote:
> this requires tcl 8.5* ?
> 
Yes. I've used dict API functions and one or two others that I later 
found out to be new in 8.5 as well. At the moment it would still not 
be too hard to change back to list functions and there are probably 
ways to do the other things with 8.4 functions as well.

But doing that would probably set the expectation that the project 
will strive to work with 8.4 for a long time to come. Since 8.5 is 
the current Tcl release, I don't feel like restricting the project 
like that. I'm currently thinking about changing the dbus command 
into an ensemble command, which would not be possible if it needs to 
stay compatible with 8.4.

Schelte.
-- 
set Reply-To [string map {nospam schelte} $header(From)]
0
nospam6520 (256)
9/6/2008 11:55:42 AM
Schelte Bron wrote:
> Uwe Klein wrote:
> 
>>this requires tcl 8.5* ?
>>
> 
> Yes. I've used dict API functions and one or two others that I later 
> found out to be new in 8.5 as well. At the moment it would still not 
> be too hard to change back to list functions and there are probably 
> ways to do the other things with 8.4 functions as well.
> 
> But doing that would probably set the expectation that the project 
> will strive to work with 8.4 for a long time to come. Since 8.5 is 
> the current Tcl release, I don't feel like restricting the project 
> like that. I'm currently thinking about changing the dbus command 
> into an ensemble command, which would not be possible if it needs to 
> stay compatible with 8.4.
I am still stuck with 8.4 on all machines.
Now my primary object of dbus use is the openmoko freerunner which
has working tcl/tk8.4.11.  But thats _my_ problem.

going to test on my Laptop first.

uwe
0
9/6/2008 3:02:05 PM
Schelte Bron wrote:
> Uwe Klein wrote:
> 
>>this requires tcl 8.5* ?
>>
> 
> Yes. I've used dict API functions and one or two others that I later 
> found out to be new in 8.5 as well. At the moment it would still not 
> be too hard to change back to list functions and there are probably 
> ways to do the other things with 8.4 functions as well.
> 
> But doing that would probably set the expectation that the project 
> will strive to work with 8.4 for a long time to come. Since 8.5 is 
> the current Tcl release, I don't feel like restricting the project 
> like that. I'm currently thinking about changing the dbus command 
> into an ensemble command, which would not be possible if it needs to 
> stay compatible with 8.4.
> 
> Schelte.
Hi,

tried to build tcldbus:
i have dbus-1-devel-1.0.2-59.4 installed.
and a current active tcl 8.5
making docs breaks on some docbook error:
	dtplite -o doc/dbus-tcl.html html doc/dbus-tcl.man
	/usr/bin/dtplite: Doctools Error in macro at line 36, column 0:
	[para]
	--> (FmtError) Manpage error (nolistcmd), "para" : \
		unknown error code "nolistcmd" (for locale posix).
this doesnt bother me much.
	
but:
requiring package dbus breaks on unresolved symbol.

ldd -d -r /opt/ActiveTcl-8.5/lib/dbus0.5/libdbus0.5.so
undefined symbol: Tcl_DictObjPut    (/opt/ActiveTcl-8.5/lib/dbus0.5/libdbus0.5.so)
undefined symbol: Tcl_ObjPrintf (/opt/ActiveTcl-8.5/lib/dbus0.5/libdbus0.5.so)
!!! undefined symbol: dbus_get_version  (/opt/ActiveTcl-8.5/lib/dbus0.5/libdbus0.5.so)
!!! undefined symbol: dbus_connection_get_server_id (/opt/ActiveTcl-8.5/lib/dbus0.5/libdbus0.5.so)
undefined symbol: Tcl_NewDictObj    (/opt/ActiveTcl-8.5/lib/dbus0.5/libdbus0.5.so)
         linux-gate.so.1 =>  (0xffffe000)
         libdbus-1.so.3 => /lib/libdbus-1.so.3 (0xb7f08000)
         libc.so.6 => /lib/libc.so.6 (0xb7dd5000)
         /lib/ld-linux.so.2 (0x80000000)

should I have a newer version of dbus?

uwe
0
9/13/2008 3:08:28 PM
Uwe Klein wrote:
> tried to build tcldbus:
> i have dbus-1-devel-1.0.2-59.4 installed.
   [snip]
> should I have a newer version of dbus?
> 
Possibly. I'm using dbus-1-devel-1.2.1-15.1

Schelte.
-- 
set Reply-To [string map {nospam schelte} $header(From)]
0
nospam6520 (256)
9/14/2008 7:33:10 PM
Schelte Bron wrote:
> Uwe Klein wrote:
> 
>>tried to build tcldbus:
>>i have dbus-1-devel-1.0.2-59.4 installed.
> 
>    [snip]
> 
>>should I have a newer version of dbus?
>>
> 
> Possibly. I'm using dbus-1-devel-1.2.1-15.1
> 
> Schelte.
I've fixed the code to not use those functions.
hardcoded the version request ...

I have a bit of a problem finding a good start.
Is there by any chance a (simple) example script?

( I currently have the impression that dbus ( the framework )
is exessively complex ( and opaq?) but does not
completely solve its problems )


uwe
0
9/14/2008 9:24:10 PM
Uwe Klein wrote:
> I have a bit of a problem finding a good start.
> Is there by any chance a (simple) example script?
> 
Here's a small client server example:

# server code:
package require dbus

dbus connect
dbus name foo.tcl.tk
dbus register -fallback / handler

proc handler {dict args} {
        if {[dict get $dict messagetype] eq "method_call"} {
                return [[dict get $dict member] {*}$args]
        }
}

vwait forever
##########################################################

# client code:
package require dbus

dbus connect

puts [dbus call -dest foo.tcl.tk / foo.tcl.tk pwd]
puts [dbus call -dest foo.tcl.tk / foo.tcl.tk info patchlevel]
dbus call -timeout -1 -dest foo.tcl.tk / foo.tcl.tk exit
##########################################################

> ( I currently have the impression that dbus ( the framework )
> is exessively complex ( and opaq?) but does not
> completely solve its problems )
> 
I welcome any suggestions you may have to improve the interface.

Schelte.
-- 
set Reply-To [string map {nospam schelte} $header(From)]
0
nospam6520 (256)
9/15/2008 7:50:06 PM
Schelte Bron wrote:
> Uwe Klein wrote:
> 
>>I have a bit of a problem finding a good start.
>>Is there by any chance a (simple) example script?
>>
> 
> Here's a small client server example:

Thank you very much.

>>( I currently have the impression that dbus ( the framework )
>>is exessively complex ( and opaq?) but does not
>>completely solve its problems )
>>
> 
> I welcome any suggestions you may have to improve the interface.

I've got to get a bit more behind the thinking in dbus.

( the openmoko phones use dbus for system management.)
> 
> Schelte.

uwe

0
9/15/2008 8:10:31 PM
Reply: