f



png data format

Hello

I'm having trouble with this imput of data from a PNG image.  The specification says that "chunks" have a 4 byte field that is the length of the attached data segment.  I tried to read the length in for a chunk that has a length of 13, which was confirmed in a hexdump

0000000 211 120 116 107 015 012 032 012 -->>000 000 000 015<<-- 111 110 104 122
0000010 000 000 041 215 000 000 007 165 010 006 000 000 001 206 055 074
0000020 336 000 000 000 004 147 101 115 101 000 000 261 217 013 374 141

I am storing the data in a uint32_t variable using the following code, but the value keeps showing up with a huge number 218103808 which happens to be the number that is evaluated by iostream for the value of the whole chunk


done reading header



Sizeof Chunk 4
Raw Chunk Number 0: 218103808
***LENGTH****
Length value => 218103808
Sizeof Byte 1
Character 0::
        ^@
Byte 0::
        0
Character 1::
        ^@
Byte 1::
        0
Character 2::
        ^@
Byte 2::
        0
Character 3::
        Byte 3::
        13


As yet, when I break it down by single bytes, it returns 0 0 0 13, which is correct.  ddd seems to say the same thing, and I don't know why.  When evaluated as 4 bytes, you get this large number, but when you evaluate them seperately, each byte, it comes out right.

The code snippet I'm using looks like this

in the .h file #ifndef PNGPRJ
#define PNGPRJ
#include <inttypes.h>
namespace png_proj{
	typedef uint32_t CHUNK;



In the .cpp file
	void Image::read_chunk()
	{
		char * cur = get_index();
		CHUNK * tmp = reinterpret_cast<CHUNK *>(cur);
		std::cout << std::endl << "Sizeof Chunk "  << sizeof(*tmp) << std::endl;
		for(int j = 0; j<4; j++){
			std::cout << "Raw Chunk Number " << j << ": " << *tmp << std::endl;
			
			
			switch ( j ) {
				case 0:
					std::cout << "***LENGTH****" << std::endl;
					set_length(static_cast<int32_t>(*tmp));
					std::cout << "Length value => " << static_cast<int>(*tmp) << std::endl;
					break;

				case 1:	
					std::cout << "***TYPE****" << std::endl;
					set_type(static_cast<int32_t>(*tmp));
					break;

				case 2:
					{
					std::cout << "***DATA****" << std::endl;
					unsigned long int l = static_cast<unsigned long int>(get_length());
					std::cout << "buffer size should be " << get_length() << std::endl;
					int8_t * buffer = new int8_t[l];
					std::cout << "buffer element size is " << *buffer  << std::endl;
					std::cout << "buffer size is " << l  << std::endl;
					for(unsigned int k = 0; k < get_length(); k++){
						buffer[k] = static_cast<int8_t>(tmp[k]);
						std::cout << "data " << *buffer  << std::endl;
					}
					set_data(buffer);
					}
					break;

				case 3:	
					std::cout << "***CRC****" << std::endl;
					set_crc(static_cast<int32_t>(*tmp));
					break;

				default:	
					std::cout << "***NOMANDSLAND****" << std::endl;
					break;
			}				/* -----  end switch  ----- */

			char * tmp2 = reinterpret_cast<char *>(tmp); //reading each byte
			std::cout << "Sizeof Byte "  << sizeof(*tmp2) << std::endl;
			//std::cout << "Mark ==>>" << __LINE__ << std::endl;
			for(int i=0; i<4; i++){
				std::cout << "Character " << i << "::" << std::endl << "\t" << *tmp2 << std::endl;
				std::cout << "Byte " << i << "::" << std::endl << "\t" <<  static_cast<unsigned long int>(*tmp2) << std::endl;
				tmp2++;
			}
			std::cout<<std::endl;
			std::cout<<std::endl;
			tmp++;
			cur = ( reinterpret_cast<char*>(tmp) );
		}
		set_index(cur);
	}



I dug through libpng since this seems to not being doing what I expected.  They seem to set it up as 4 byte array

void /* PRIVATE */
png_push_read_chunk(png_structrp png_ptr, png_inforp info_ptr)
{
   png_uint_32 chunk_name;
#ifdef PNG_HANDLE_AS_UNKNOWN_SUPPORTED
   int keep; /* unknown handling method */
#endif

   /* First we make sure we have enough data for the 4-byte chunk name
    * and the 4-byte chunk length before proceeding with decoding the
    * chunk data.  To fully decode each of these chunks, we also make
    * sure we have enough data in the buffer for the 4-byte CRC at the
    * end of every chunk (except IDAT, which is handled separately).
    */
   if ((png_ptr->mode & PNG_HAVE_CHUNK_HEADER) == 0)
   {
      png_byte chunk_length[4];
      png_byte chunk_tag[4];

      PNG_PUSH_SAVE_BUFFER_IF_LT(8)
      png_push_fill_buffer(png_ptr, chunk_length, 4);
      png_ptr->push_length = png_get_uint_31(png_ptr, chunk_length);
      png_reset_crc(png_ptr);
      png_crc_read(png_ptr, chunk_tag, 4);
      png_ptr->chunk_name = PNG_CHUNK_FROM_STRING(chunk_tag);
      png_check_chunk_name(png_ptr, png_ptr->chunk_name);
      png_ptr->mode |= PNG_HAVE_CHUNK_HEADER;
   }


I'm obviously not understanding something I'm evaluation here.  So I'm wondering if anyone can shed light on this.
http://www.nylxs.com/docs/grad_school/parallel/src/png/png_proj.h
http://www.nylxs.com/docs/grad_school/parallel/src/png/png_proj.cpp
http://www.nylxs.com/docs/grad_school/parallel/src/png/main_png.cpp
http://www.nylxs.com/docs/grad_school/parallel/src/png/makefile

ruben

let.me.in


Ruben
0
ruben
12/6/2016 2:08:58 PM
comp.unix.programmer 10848 articles. 0 followers. kokososo56 (349) is leader. Post Follow

5 Replies
170 Views

Similar Articles

[PageSpeed] 5

On 12/6/2016 9:08 AM, ruben safir wrote:
> Hello
>
> I'm having trouble with this imput of data from a PNG image.  The specification says that "chunks" have a 4 byte field that is the length of the attached data segment.  I tried to read the length in for a chunk that has a length of 13, which was confirmed in a hexdump
>
> 0000000 211 120 116 107 015 012 032 012 -->>000 000 000 015<<-- 111 110 104 122

     Odd to term this a "hexdump," since it appears to be octal ...
In hexadecimal, the highlighted bytes would be

	00 00 00 0D

> I am storing the data in a uint32_t variable using the following code, but the value keeps showing up with a huge number 218103808

     What's that huge number, expressed in hexadecimal?  It's

	0D 00 00 00

Now: Can you divine any particular relation between this value and the
value from your "hexdump?"  If not, here's a hint:

https://en.wikipedia.org/wiki/Endianness

-- 
esosman@comcast-dot-net.invalid
"Nobody ever went broke underestimating the intelligence of the
American public." -- HLM (paraphrased)
0
Eric
12/6/2016 2:26:19 PM
ruben safir <ruben@mrbrklyn.com> wrote:
> I'm having trouble with this imput of data from a PNG image. The
> specification says that "chunks" have a 4 byte field that is the length of
> the attached data segment. I tried to read the length in for a chunk that
> has a length of 13, which was confirmed in a hexdump

> 0000000 211 120 116 107 015 012 032 012 -->>000 000 000 015<<-- 111 110 104 122
> 0000010 000 000 041 215 000 000 007 165 010 006 000 000 001 206 055 074
> 0000020 336 000 000 000 004 147 101 115 101 000 000 261 217 013 374 141

> I am storing the data in a uint32_t variable using the following code, but
> the value keeps showing up with a huge number 218103808 which happens to be
> the number that is evaluated by iostream for the value of the whole chunk


> done reading header

> Sizeof Chunk 4
> Raw Chunk Number 0: 218103808
> ***LENGTH****
> Length value => 218103808

The data are stored in big-endian order, i.e. with the most
significant byte coming first, but you seem to have read it
in as a 32-bit integer on a machine using little-endian byte
order, e.g. with a machine with an Intel processor, where
the least-significant byte comes first in memory. That's
also what they expect when reading in binary data from a
file. So simply read in the four bytes separately into e.
g. an array

   uint8_t bytes[4]

with the first byte you read from the file going into
the first array element etc. Then do

uint32_t len = ((bytes[0] * 256 + bytes[1]) * 256 + bytes[2]) * 256 + bytes[3];

and you should get the correct value. The same applies, of
course, also for other fields containing values that occupy
multiple bytes! This then will work correctly on both big-
and little-endian machines.
                             Regards, Jens
-- 
  \   Jens Thoms Toerring  ___      jt@toerring.de
   \__________________________      http://toerring.de
0
jt
12/7/2016 7:49:12 PM
Jens Thoms Toerring wrote:
> ruben safir <ruben@mrbrklyn.com> wrote:
>> I'm having trouble with this imput of data from a PNG image. 

[...]
>
> The data are stored in big-endian order, i.e. with the most
> significant byte coming first, but you seem to have read it
> in as a 32-bit integer on a machine using little-endian byte
> order, e.g. with a machine with an Intel processor, where
> the least-significant byte comes first in memory.

I'm too lazy to look it up now, and it has been a while since I used the
library, but I'm certain that libpng contains functions to portably deal
with this situation. Or are you for some reason forced to go make do without
libpng?

robert
0
Robert
12/9/2016 6:31:43 PM
On Fri, 09 Dec 2016 18:31:43 +0000, Robert Latest wrote:

> Or are you for some reason forced to go make do without libpng?

yeah
0
Popping
12/15/2016 4:13:58 AM
On 12/14/2016 11:13 PM, Popping mad wrote:
> On Fri, 09 Dec 2016 18:31:43 +0000, Robert Latest wrote:
> 
>> Or are you for some reason forced to go make do without libpng?
> 
> yeah
> 

For reference purposes:

~~~~~~~~~`
On Mon, Dec 12, 2016 at 5:45 PM, Ruben Safir <ruben@mrbrklyn.com> wrote:
> So now I have a question about the data in the IDAT chuncks.
>
> do they likiwise need to be reversed in the byte order before sending
> them to be decompressed with zlib?

The zlib specification defines the compressed data as a *byte* stream.
This is then split into sections (arbitrarily, except that each
section must be less than 2^31 bytes long) and sections are stored in
a sequence of IDAT chunks (no intervening chunks permitted).

The only 8/32 bit issue here is that each *section* is checksumed
using CRC32 (the cyclic redundancy code defined for Ethernet packets)
and, while this also checksums a *byte* stream, the result is a 32-bit
number which is encoded at the end of each chunk as a big-endian
sequence of four bytes.  This is exactly the same as the check sum on
every other chunk.

Internally the zlib stream is defined to end with an Adler32 checksum.
This is another 32-bit checksum of a sequence of bytes but now over
the whole data stream *before* compression.  This is, in turn, encoded
as a sequence of 4 bytes at the end of the *compressed* stream but
that definition comes from the zlib spec.  IRC it is big-endian (i.e.
the same as the CRC32 that immediately follows it.)

Anyway, the internals of zlib are not part of the PNG spec and they
have to deal with the simple fact that the zlib stream (well, the
deflate sub-part) is actually a *bit* stream that has to be mapped to
*bytes* to make a byte stream.

Roll-your-own zlib decompression is, in fact, very straight-forward
however nowhere near as simple as decoding a PNG stream using an
existing decompression implementation.

The relevant references are, high level to low:

PNG specification: http://www.libpng.org/pub/png/spec/iso/
    (Glenn gave an equivalent link to the W3C web site; I prefer the
ISO spec because it is the normative version, it also has lots of nice
explanation and pretty pictures.)
Zlib specification: https://www.ietf.org/rfc/rfc1950.txt
    Defines a big-endian multi-byte format
Deflate specification: https://www.ietf.org/rfc/rfc1951.txt
    Defines a little-endian multi-byte format

Notice that the zlib and deflate specifications use different ordering
for multi-byte numbers.  It is important to read the specification and
to read the *correct* specification when decoding data!  Indeed, as
the deflate specification points out, different numbers within the
different parts of the spec are encoded in different ways; in some
cases numbers are written as Huffman codes and, in that case, the most
significant parts of the Huffman code are written first; the code is
effectively big-endian (although that description has its own set of
assumptions.)

John Bowler

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
png-mng-implement mailing list
png-mng-implement@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/png-mng-implement


0
ruben
12/15/2016 8:33:55 PM
Reply: