Is there a differential archiving format which works well with random access

  • Follow


I am looking for a format to archive multiple images. The images have a spe=
cial property in that: often series of images will be very similar to each =
other. The whole image set will have several of these groups (but each grou=
p will be quite different to each other).

I am looking for an archiving format which takes advantage of differential =
storage, but also have a good random access rate.

My conceptual format works like this. Say there are three files, A, A' and =
A''. They are quite similar to each other. The archive will store A, and th=
e difference A-A', and the difference A-A''.

When accessing A'', it will access A, and then access the difference A-A''.=
 It will output the A merged with that difference.

Since it requires random access, the archive will store the file offset inf=
ormation in an index section.

I am wondering if there already exists a format with similar capabilities.
0
Reply mcha226 (11) 8/7/2012 2:38:00 PM

w <mcha226@gmail.com> wrote:

> I am looking for a format to archive multiple images. 
> The images have a special property in that: often series 
> of images will be very similar to each other. 
> The whole image set will have several of these groups 
> (but each group will be quite different to each other).

At some point, that is what MPEG does.

It considers a movie as groups of images that are similar and,
for each group, includes one copy of the whole image and then
differences, including various translations (in space).

While MPEG might not be right for you, it might be a good
starting point.

-- glen
0
Reply gah (12259) 8/7/2012 3:41:29 PM


glen herrmannsfeldt=E6=96=BC 2012=E5=B9=B48=E6=9C=887=E6=97=A5=E6=98=9F=E6=
=9C=9F=E4=BA=8CUTC+8=E4=B8=8B=E5=8D=8811=E6=99=8241=E5=88=8629=E7=A7=92=E5=
=AF=AB=E9=81=93=EF=BC=9A
> At some point, that is what MPEG does.
> It considers a movie as groups of images that are similar and,
> for each group, includes one copy of the whole image and then
> differences, including various translations (in space).
> While MPEG might not be right for you, it might be a good
> starting point.
> -- glen

I think you are quite right. They are quite similar. After reading the wiki=
pedia, I think the differences are that the differentials are based on adja=
cent frames but not the key frame, and it is not lossless.

I guess my dream format is waiting to be invented.
0
Reply mcha226 (11) 8/8/2012 12:26:36 PM

Am 07.08.2012 16:38, schrieb w:
> I am looking for a format to archive multiple images. The images have a special property in that: often series of images will be very similar to each other. The whole image set will have several of these groups (but each group will be quite different to each other).
>
> I am looking for an archiving format which takes advantage of differential storage, but also have a good random access rate.
>
> My conceptual format works like this. Say there are three files, A, A' and A''. They are quite similar to each other. The archive will store A, and the difference A-A', and the difference A-A''.
>
> When accessing A'', it will access A, and then access the difference A-A''. It will output the A merged with that difference.
>
> Since it requires random access, the archive will store the file offset information in an index section.
>
> I am wondering if there already exists a format with similar capabilities.

Certainly. JPEG 2000 part 2 with a wavelet transformation or a 
differential color transformation in the third direction would easily do 
that for you. Not that I'm aware of many implementations (I have one you 
can get from Accusoft) but it can do pretty much of what you describe.

Greetings,
	Thomas


0
Reply thor16 (320) 8/8/2012 1:37:11 PM

Thomas Richter=E6=96=BC 2012=E5=B9=B48=E6=9C=888=E6=97=A5=E6=98=9F=E6=9C=9F=
=E4=B8=89UTC+8=E4=B8=8B=E5=8D=889=E6=99=8237=E5=88=8611=E7=A7=92=E5=AF=AB=
=E9=81=93=EF=BC=9A
> Am 07.08.2012 16:38, schrieb w:
>=20
> > I am looking for a format to archive multiple images. The images have a=
 special property in that: often series of images will be very similar to e=
ach other. The whole image set will have several of these groups (but each =
group will be quite different to each other).
>=20
> >
>=20
> > I am looking for an archiving format which takes advantage of different=
ial storage, but also have a good random access rate.
>=20
> >
>=20
> > My conceptual format works like this. Say there are three files, A, A' =
and A''. They are quite similar to each other. The archive will store A, an=
d the difference A-A', and the difference A-A''.
>=20
> >
>=20
> > When accessing A'', it will access A, and then access the difference A-=
A''. It will output the A merged with that difference.
>=20
> >
>=20
> > Since it requires random access, the archive will store the file offset=
 information in an index section.
>=20
> >
>=20
> > I am wondering if there already exists a format with similar capabiliti=
es.
>=20
>=20
>=20
> Certainly. JPEG 2000 part 2 with a wavelet transformation or a=20
>=20
> differential color transformation in the third direction would easily do=
=20
>=20
> that for you. Not that I'm aware of many implementations (I have one you=
=20
>=20
> can get from Accusoft) but it can do pretty much of what you describe.
>=20
>=20
>=20
> Greetings,
>=20
> 	Thomas

I have just had a look. JPEG2000 part 6 Compound jpm format (http://www.dig=
italpreservation.gov/formats/fdd/fdd000144.shtml) seems like the one which =
fits my condition. But you mentioned part 2, which seems to be a single ima=
ge format. Does it allows for two set of images, each set containing simila=
r images, but two sets are very different? Thanks.
0
Reply mcha226 (11) 8/9/2012 4:02:18 PM

	Thomas
>
> I have just had a look. JPEG2000 part 6 Compound jpm format (http://www.digitalpreservation.gov/formats/fdd/fdd000144.shtml) seems like the one which fits my condition. But you mentioned part 2, which seems to be a single image format. Does it allows for two set of images, each set containing similar images, but two sets are very different? Thanks.

Please format your postings properly, thanks.

Yes, part 2 allows that, too. It may contain multiple images (there 
called compositing layers), but it does not allow images mixed with 
bi-level graphics (in a reasonable way at least) as does part-6.

BTW, I wouldn't expect a high advantage of running an additional 
redundancy reduction over he images. The high-passes are already pretty 
much decorrelated, and the single low-pass that is left is the only part 
of the image that can profit from such a decorrelation transformation. 
But this is the smallest part of the image, so there is not too much gain.

If the images are correlated like in a movie/video, then you probably 
migh want to look into H.264/HEVC.

Greetings,
	Thomas
0
Reply thor16 (320) 8/11/2012 5:01:24 AM

5 Replies
32 Views

(page loaded in 0.215 seconds)


Reply: