Hello c.p.d.,

in a current project, I am looking for an Import Application for
clinical use that imports non-DICOM scan Images to a DICOM PACS.

I could not find a suitable Software by now - but I think it would be
something very useful to have, and so I am considering if an Open Source
 Project to build such an application might be feasible.

But I came across a major concern: For clinical use, the project has to
make sure that UIDs are unique - as it is intended for clinical use,
this is absolutely crucial.

The application itself can make sure it generates UIDs unique in its own
application context, e. g. by using timestamps or counters and avoid
multithreaded generation on UIDs in parallel. But two Application on two
boxes might be a problem there if they do not use a
Installation-specific UID root.

So before teaming up to start such a project, I would like to discuss
with you (especially the ones that are already maintaining Open Source
toolkits and applications) if it is a good idea to provide an
application generating real-world objects for clinical use with an UID
generation mechanism that would require

* a user to provide a UID root of his own
* a user to keep track of all installations and maker sure every single
installation uses a unique UID root
* some person or institution acting as the warranter for UID uniqueness.

I know there are free UID root providers, as well as I know UID roots
can be obtained easily in some countries (not in high tech Germany, alas)...

Some ideas I would like to have your comments on:

a) provide a UID Root like "INVALID_ROOT" as UID Root by default

b) provide an extensive documentation about the issue and instruct the
user either to do it properly or else ask a DICOM aware consultant of
his trust to do it right

c) create a "register UID root for your Installation" service on the
project's website, delivering a cipher upon successful registration that
removes "NOT FOR CLINICAL USE" warning in the application (hard to make
sure this does not get patched away, however) for the root provided
(still, the once registered root can be re-used all over the world if
spread). This is quite expensive, however.

d) not do it at all.

I am afraid that the application would bear some potential danger if
misused and I do not want to be  the one to be made responsible for the
mess.

So what do you think? What are your experiences?

If my concerns can be resolved to my entire satisfaction, I will pick up
the idea and take next steps, including the acquisition of contributors
and sponsors.

But if I see a can of worms will be opened by this (with "Pandora"
written on it ;o) ), I have to re-consider to maybe make the application
proprietary or buy something proprietary commercial instead.


So please comment or provide your personal experiences. Maybe there is a
easy solution that did not yet come to my mind (he typed hopefully ;o) )

I am curious to hear from you!


Regards,


Peter

Peter,

The ClearCanvas DICOM library does contain a valid UID generator in
it, although
its not really advertised.  The ClearCanvas Workstation's
anonymization
routines use this UID generator.

We came up with a scheme that uses our base UID, the MAC address of
the
ethernet card in the system converted into a decimal, a timestamp, a
process ID,
 and a counter.  We had to be careful because we came up close to the
64
character UID limit, but it is a valid UID generator.  We're dependent
on people
not changing their MAC addresses for uniqueness, but we believe that's
a
reasonable assumption.

FYI, you can get a base UID for free from IANA, which we used,
here:

http://pen.iana.org/pen/PenApplication.page

I would be a bit reluctant to put out a program that depends on users
to
enter/acquire a base UID.  Although you could reference the above
site, the chance of a user putting in something like "1.1.1.1.1" for
the base
UID to get the product operational is fairly high.  I don't think it
would be
worth the effort to bother with registration schemes, etc., in the
product
itself.

Regards,
Steve

On Nov 23, 7:21=A0am, Peter B Schmidt <n...@PBSchmidt.de> wrote:
> Hello c.p.d.,
>
> in a current project, I am looking for an Import Application for
> clinical use that imports non-DICOM scan Images to a DICOM PACS.
>
> I could not find a suitable Software by now - but I think it would be
> something very useful to have, and so I am considering if an Open Source
> =A0Project to build such an application might be feasible.
>
> But I came across a major concern: For clinical use, the project has to
> make sure that UIDs are unique - as it is intended for clinical use,
> this is absolutely crucial.
>
> The application itself can make sure it generates UIDs unique in its own
> application context, e. g. by using timestamps or counters and avoid
> multithreaded generation on UIDs in parallel. But two Application on two
> boxes might be a problem there if they do not use a
> Installation-specific UID root.
>
> So before teaming up to start such a project, I would like to discuss
> with you (especially the ones that are already maintaining Open Source
> toolkits and applications) if it is a good idea to provide an
> application generating real-world objects for clinical use with an UID
> generation mechanism that would require
>
> * a user to provide a UID root of his own
> * a user to keep track of all installations and maker sure every single
> installation uses a unique UID root
> * some person or institution acting as the warranter for UID uniqueness.
>
> I know there are free UID root providers, as well as I know UID roots
> can be obtained easily in some countries (not in high tech Germany, alas)=
....
>
> Some ideas I would like to have your comments on:
>
> a) provide a UID Root like "INVALID_ROOT" as UID Root by default
>
> b) provide an extensive documentation about the issue and instruct the
> user either to do it properly or else ask a DICOM aware consultant of
> his trust to do it right
>
> c) create a "register UID root for your Installation" service on the
> project's website, delivering a cipher upon successful registration that
> removes "NOT FOR CLINICAL USE" warning in the application (hard to make
> sure this does not get patched away, however) for the root provided
> (still, the once registered root can be re-used all over the world if
> spread). This is quite expensive, however.
>
> d) not do it at all.
>
> I am afraid that the application would bear some potential danger if
> misused and I do not want to be =A0the one to be made responsible for the
> mess.
>
> So what do you think? What are your experiences?
>
> If my concerns can be resolved to my entire satisfaction, I will pick up
> the idea and take next steps, including the acquisition of contributors
> and sponsors.
>
> But if I see a can of worms will be opened by this (with "Pandora"
> written on it ;o) ), I have to re-consider to maybe make the application
> proprietary or buy something proprietary commercial instead.
>
> So please comment or provide your personal experiences. Maybe there is a
> easy solution that did not yet come to my mind (he typed hopefully ;o) )
>
> I am curious to hear from you!
>
> Regards,
>
> Peter

Hello Steve,

thank you for your recommendations.

Steve wrote:
> We came up with a scheme that uses our base UID, the MAC address of
> the
> ethernet card in the system converted into a decimal, a timestamp, a
> process ID,
>  and a counter.  We had to be careful because we came up close to the
> 64
> character UID limit, but it is a valid UID generator.

Using the MAC is one possible solution (though I heard from Network
Administrators that had to struggle with duplicate MAC in their network
already - very very improbable, but okay).


With millisecond timestamp and counter, you should be fine, but does
your UID Generator run as a singleton? Imagine two threads in the same
millisecond, before updating the counter...

Well, okay, very very improbable as well.

I only wonder if it is always possible to get your adapter's MAC if you
use JAVA (which would be my favorite as it is platfor independent)...
And additionally, if I take a 6 byte MAC, this would make 3 WORD of
integer which can take 281474976710656 different values ... that's 15
valuable character off the 64 char limit...

Anonymization is a rather rare use case, I am a bit intimidated by the
frequent "generate Images" use case. Frankly, if it was for
anonymization, I would not have the concerns detailed in my OP, as
anonymized instances are a bit out of clinical focus.

Thanks for your Remarks, maybe I'll investigate a bit more in this
direction. Your comments are very much appreciated!


Have a nice day,


Peter

On Nov 23, 5:04=A0pm, Peter B Schmidt <n...@PBSchmidt.de> wrote:
> Hello Steve,
>
> thank you for your recommendations.
>
> Steve wrote:
> > We came up with a scheme that uses our base UID, the MAC address of
> > the
> > ethernet card in the system converted into a decimal, a timestamp, a
> > process ID,
> > =A0and a counter. =A0We had to be careful because we came up close to t=
he
> > 64
> > character UID limit, but it is a valid UID generator.
>
> Using the MAC is one possible solution (though I heard from Network
> Administrators that had to struggle with duplicate MAC in their network
> already - very very improbable, but okay).
>
> With millisecond timestamp and counter, you should be fine, but does
> your UID Generator run as a singleton? Imagine two threads in the same
> millisecond, before updating the counter...
>
> Well, okay, very very improbable as well.
>
> I only wonder if it is always possible to get your adapter's MAC if you
> use JAVA (which would be my favorite as it is platfor independent)...
> And additionally, if I take a 6 byte MAC, this would make 3 WORD of
> integer which can take 281474976710656 different values ... that's 15
> valuable character off the 64 char limit...
>
> Anonymization is a rather rare use case, I am a bit intimidated by the
> frequent "generate Images" use case. Frankly, if it was for
> anonymization, I would not have the concerns detailed in my OP, as
> anonymized instances are a bit out of clinical focus.
>
> Thanks for your Remarks, maybe I'll investigate a bit more in this
> direction. Your comments are very much appreciated!

Peter,

  I know this can be very fun to implement your own UID generator, but
I would strongly suggest against doing so. Most UNIX system/Win32
comes with a very good implementation of UUID (Unique UID). If you
want something portable, look at my implementation in GDCM (*).
  Trying to read the MAC adress is a very bad idea, some people
(military, or very paranoid), run executable in some kind of
environment where this value cannot be read, or if a program tries to
access it is consider unsafe program, and may be disallowed.

  You should really read the open source implementation of libuuid
(**), this is very informative. The only real problem you have to deal
with is that if your Root UID eats up too much space, and you cannot
store the full 128bits random uuid in the remaining bits of the VR:UI
attribute (see also in gdcm::UIDGenerator how I pack those 128bits).

> anonymized instances are a bit out of clinical focus.

  Anonymization is the very first of any Clinical Trial environement !
People do mistake, and when you receive non-anonymized data, you
should still be ready to anonymize the data, keep the lookup of old
UID to new UID. And then distribute the anonymized data to your team.
Only a couple of people should be allowed to have access to the non-
anonymized UID, to be able to contact the patient if anything gets
detected during a Clinical Trial session.

Cheers
-Mathieu
(*) http://gdcm.sourceforge.net/html/classgdcm_1_1UIDGenerator.html
(**) http://gdcm.svn.sf.net/viewvc/gdcm/trunk/Utilities/gdcmuuid/

Mathieu Malaterre wrote:
....
>   I know this can be very fun to implement your own UID generator, but
> I would strongly suggest against doing so. Most UNIX system/Win32
> comes with a very good implementation of UUID (Unique UID). If you
> want something portable, look at my implementation in GDCM (*).

Speaking of UID production from UUIDs, there is a standard and
reversible way to do this described in ITU X.667, and I discuss
it more in my FAQ, at:

http://www.dclunie.com/medical-image-faq/html/part2.html#UUID

Peter B Schmidt wrote:

> a) provide a UID Root like "INVALID_ROOT" as UID Root by default

Please do not write software that allows users to generate invalid
and unusable DICOM images.

David

Hello Mathieu,

thank you for your post - I see my posting was really a good idea, where
else on earth one can find so many excellent reviewers of a concept but
here ;o) ?


Mathieu Malaterre wrote:

>   You should really read the open source implementation of libuuid
> (**), this is very informative. The only real problem you have to deal
> with is that if your Root UID eats up too much space, and you cannot
> store the full 128bits random uuid in the remaining bits of the VR:UI
> attribute (see also in gdcm::UIDGenerator how I pack those 128bits).

Great, I will have a look at this - as I already said, I would prefer a
platform idependent Solution aka JAVA (the currently targeted project
uses them all: Windows, MacOS, Linux, Unices), but if I get the concept,
it should be able to be ported - with your kind agreement assumed. Thank
you for the hint - I am a lazy person, so I much appreciate using
existing solutions instead of re-inventing the wheel and hope it will be
round this time again.


>   Anonymization is the very first of any Clinical Trial environement !

I got your point. Still, Anonymization (or more precisely for our use
case, Pseudonymization) IMHO is not the "regular" use case in a
practitioner's shop.

Trials, of course, are some different kind of bird -  my focus is a
medical practitioner with some 3+ modalities.

> People do mistake, and when you receive non-anonymized data, you
> should still be ready to anonymize the data, keep the lookup of old
> UID to new UID. And then distribute the anonymized data to your team.
> Only a couple of people should be allowed to have access to the non-
> anonymized UID, to be able to contact the patient if anything gets
> detected during a Clinical Trial session.

Well said! I keep the quote although it is not in my use case, as this
is an important point, indeed.

> 
> Cheers
> -Mathieu
> (*) http://gdcm.sourceforge.net/html/classgdcm_1_1UIDGenerator.html
> (**) http://gdcm.svn.sf.net/viewvc/gdcm/trunk/Utilities/gdcmuuid/

Thank you for your hints - and for sharing gdcm with us, keep it up!

Yours,


Peter

Hello David,

thank you for your reply. Things get much clearer by now.

David Clunie wrote:
> Speaking of UID production from UUIDs, there is a standard and
> reversible way to do this described in ITU X.667, and I discuss
> it more in my FAQ, at:
> 
> http://www.dclunie.com/medical-image-faq/html/part2.html#UUID

He who can read may gain wisdom rapidly... :oD I should re-visit your
FAQ pages more frequently, this one has slipped my attention by now. So
there is a Standard, so I should *use* this Standard. (<- this is a full
stop there)

> Peter B Schmidt wrote:
>> a) provide a UID Root like "INVALID_ROOT" as UID Root by default
> 
> Please do not write software that allows users to generate invalid
> and unusable DICOM images.

This is exactly why i was begging for your comments and reviews here: I
felt uncomfortable with this and I *knew* there must be a better way,
and I think I got close to finding it.

Thank you very much, and stay tuned for more about this project - I am
sure I will need community help not only from c.p.d., but also from Java
Gurus and UI wizards. Let me bring the stones and put up the kettle (*),
and we will see where we go from here :o)


Thank you!


Yours,

Peter
-- 
(*): http://www.extremelinux.info/stonesoup/stonesoup.html

On Nov 23, 7:22=A0pm, David Clunie <dclu...@dclunie.com> wrote:
> Mathieu Malaterre wrote:
>
> ...
>
> > =A0 I know this can be very fun to implement your own UID generator, bu=
t
> > I would strongly suggest against doing so. Most UNIX system/Win32
> > comes with a very good implementation of UUID (Unique UID). If you
> > want something portable, look at my implementation in GDCM (*).
>
> Speaking of UID production from UUIDs, there is a standard and
> reversible way to do this described in ITU X.667, and I discuss
> it more in my FAQ, at:
>
> http://www.dclunie.com/medical-image-faq/html/part2.html#UUID

To avoid some confusion, libuuid is a C implementation of ITU X.667.

Ref:
http://en.wikipedia.org/wiki/Universally_Unique_Identifier


2cts
-Mathieu

Hello Mathieu,

Mathieu Malaterre wrote:
>
> To avoid some confusion, libuuid is a C implementation of ITU X.667.

thank you for the information, I googled myself and almost came up to
the same result ;o)

So, if I got it right, your suggestion is to use
<root>.<projectID>.<uuidToOID>

where <root> is "my" UID Root
<project> is an identifier for the project
[ => both make up a Project specific UID root ]
and
<uuidToOID> is an OID encoded random UUID (type 4)

Did I summarize this correctly?

Or do you just use a type 4 UUID and encode it as OOID? I think, this is
not allowed as the UID root is required to find the responsible person /
organization if there is a problem... And I do not want to hijack gdcm's
responsibility when porting to java...

Please correct me if I am wrong.

Now, the next thing is to set up the project in the right place - I
consider using a toolkit (like dcm4che, as it is JAVA and FOSS), I will
inquire if this has something similar built in already and the license fits.

Stay tuned for more ;o)

Kind regards,


Peter

On Nov 24, 10:19=A0am, Peter B Schmidt <n...@PBSchmidt.de> wrote:
> Hello Mathieu,
>
> Mathieu Malaterre wrote:
>
> > To avoid some confusion, libuuid is a C implementation of ITU X.667.
>
> thank you for the information, I googled myself and almost came up to
> the same result ;o)
>
> So, if I got it right, your suggestion is to use
> <root>.<projectID>.<uuidToOID>

Indeed. I even save one byte doing:

<root>.<projectID><uuidToOID>

This saves me the trouble of handling of the '0' at begining of
<uuidToOID> :)

> where <root> is "my" UID Root
> <project> is an identifier for the project
> [ =3D> both make up a Project specific UID root ]
> and
> <uuidToOID> is an OID encoded random UUID (type 4)
>
> Did I summarize this correctly?

exactly.

> Or do you just use a type 4 UUID and encode it as OOID? I think, this is
> not allowed as the UID root is required to find the responsible person /
> organization if there is a problem... And I do not want to hijack gdcm's
> responsibility when porting to java...

No the <root> should always be easily readable.

> Now, the next thing is to set up the project in the right place - I
> consider using a toolkit (like dcm4che, as it is JAVA and FOSS), I will
> inquire if this has something similar built in already and the license fi=
ts.

Well GDCM is also available from Java (and also Python and C#). But it
is a little less portable than a pure Java solution.
I would also consider checking PixelMed.

Cheers

Hi Peter

Peter B Schmidt wrote:

> So, if I got it right, your suggestion is to use
> <root>.<projectID>.<uuidToOID>

This is NOT necessary.

As long as you convert the UUID to a large single decimal
integer string (with no leading zeroes), EVERYBODY can use
"2.25" as the root.

You do NOT need your own root, nor a project disambiguator,
because the UUID is globally unique.

Another nice thing about this is that it is reversible; i.e.,
you can recover the UUID easily.

David

On Nov 24, 10:19=A0am, Peter B Schmidt <n...@PBSchmidt.de> wrote:
> Hello Mathieu,
>
> Mathieu Malaterre wrote:
>
> > To avoid some confusion, libuuid is a C implementation of ITU X.667.
>
> thank you for the information, I googled myself and almost came up to
> the same result ;o)
>
> So, if I got it right, your suggestion is to use
> <root>.<projectID>.<uuidToOID>
>
> where <root> is "my" UID Root
> <project> is an identifier for the project
> [ =3D> both make up a Project specific UID root ]
> and
> <uuidToOID> is an OID encoded random UUID (type 4)
>
> Did I summarize this correctly?

Just take care, that <uuidToOID> contains up to 39 digits. Therefore
there are only 24 characters left for <root>[.<projectID>].

>
> Or do you just use a type 4 UUID and encode it as OOID? I think, this is
> not allowed as the UID root is required to find the responsible person /
> organization if there is a problem... And I do not want to hijack gdcm's
> responsibility when porting to java...
>
> Please correct me if I am wrong.
>
> Now, the next thing is to set up the project in the right place - I
> consider using a toolkit (like dcm4che, as it is JAVA and FOSS), I will
> inquire if this has something similar built in already and the license fi=
ts.

Just updated dcm4che2' s UIDUtils.createUID() accordingly: s.
http://www.dcm4che.org/jira/browse/DCM-348

It only takes ~20 lines of java code:

    public static String createUID() {
        return doCreateUID(root);
    }

    public static String createUID(String root) {
        verifyUIDRoot(root);
        return doCreateUID(root);
    }

    private static String doCreateUID(String root) {
        UUID uuid =3D UUID.randomUUID();
        byte[] b17 =3D new byte[17];
        fill(b17, 1, uuid.getMostSignificantBits());
        fill(b17, 9, uuid.getLeastSignificantBits());
        String uuidStr =3D new BigInteger(b17).toString();
        int rootLength =3D root.length();
        int uuidLength =3D uuidStr.length();
        char[] dst =3D new char[rootLength + 1 + uuidLength];
        root.getChars(0, rootLength, dst, 0);
        dst[rootLength] =3D '.';
        uuidStr.getChars(0, uuidLength, dst, rootLength + 1);
        return new String(dst);
    }

    private static void fill(byte[] bb, int off, long val) {
        for (int i =3D off, shift =3D 56; shift >=3D 0; i++, shift -=3D 8)
            bb[i] =3D (byte) (val >>> shift);
    }
--
gunter


>
> Stay tuned for more ;o)
>
> Kind regards,
>
> Peter