matGetVariable unable to read large .mat files on Win7 64-bit?

  • Follow


I am using the matGetVariable to load .mat files in my C++ code. The use is plain: matOpen the file, read in the variable, close the file.

The problem is that it looks to fail on larger .mat files in a platform-specific way. Failing means either returning 0 on the mxArray, or plain segfault on an internal miGetItem() call in libmx.dll (thats how far the debugger can go). 

The fail is consistent on Win7 64-bit. I have a series of data files and it works fine upto the size of about 20MB, which has a size about 84MB when unpacked in v6 format, but fails on some variables when reading from the file if it is larger than this.

Exactly the same piece of code works perfectly in XP64 on the same data, and never fails. The installed MATLAB versions, the non-OS specific loaded dlls are exactly the same on the two machines.

I have installed both 64 and 32-bit MATLABs and my app is 32-bit (using Qt), using the 32-bit libraries and MinGW 4.5, with the libeng.dll, libmat.dll, libmx.dll ported to minGW format via pexports and dlltool.

I am posting this to get information whether somebody else met with it, or everybody else are happily using matGetVariable() on Win7 64-bit without errors on large files.
0
Reply Balint 12/3/2010 3:58:04 AM


"Balint Takacs" <b.takacs@imperial.ac.uk> wrote in message 
news:id9psc$t5b$1@fred.mathworks.com...
> I am using the matGetVariable to load .mat files in my C++ code. The use 
> is plain: matOpen the file, read in the variable, close the file.
>
> The problem is that it looks to fail on larger .mat files in a 
> platform-specific way. Failing means either returning 0 on the mxArray, or 
> plain segfault on an internal miGetItem() call in libmx.dll (thats how far 
> the debugger can go).
> The fail is consistent on Win7 64-bit. I have a series of data files and 
> it works fine upto the size of about 20MB, which has a size about 84MB 
> when unpacked in v6 format, but fails on some variables when reading from 
> the file if it is larger than this.
>
> Exactly the same piece of code works perfectly in XP64 on the same data, 
> and never fails. The installed MATLAB versions, the non-OS specific loaded 
> dlls are exactly the same on the two machines.
>
> I have installed both 64 and 32-bit MATLABs and my app is 32-bit (using 
> Qt), using the 32-bit libraries and MinGW 4.5, with the libeng.dll, 
> libmat.dll, libmx.dll ported to minGW format via pexports and dlltool.
>
> I am posting this to get information whether somebody else met with it, or 
> everybody else are happily using matGetVariable() on Win7 64-bit without 
> errors on large files.

Please send a small sample of C++ code with which you can reproduce the 
problem, a sample MAT-file on which that sample C++ code fails, and the 
segmentation violation log file to Technical Support so that they can 
investigate whether the problem is a corrupted MAT-file, an error in your 
usage of matGetVariable, a bug in matGetVariable, or something else.

-- 
Steve Lord
slord@mathworks.com
comp.soft-sys.matlab (CSSM) FAQ: http://matlab.wikia.com/wiki/FAQ
To contact Technical Support use the Contact Us link on 
http://www.mathworks.com 

0
Reply Steven_Lord 12/3/2010 3:08:32 PM


"Balint Takacs" wrote in message <id9psc$t5b$1@fred.mathworks.com>...
> I am using the matGetVariable to load .mat files in my C++ code. The use is plain: matOpen the file, read in the variable, close the file.
> 
> The problem is that it looks to fail on larger .mat files in a platform-specific way. Failing means either returning 0 on the mxArray, or plain segfault on an internal miGetItem() call in libmx.dll (thats how far the debugger can go). 
> 
> The fail is consistent on Win7 64-bit. I have a series of data files and it works fine upto the size of about 20MB, which has a size about 84MB when unpacked in v6 format, but fails on some variables when reading from the file if it is larger than this.
> 
> Exactly the same piece of code works perfectly in XP64 on the same data, and never fails. The installed MATLAB versions, the non-OS specific loaded dlls are exactly the same on the two machines.
> 
> I have installed both 64 and 32-bit MATLABs and my app is 32-bit (using Qt), using the 32-bit libraries and MinGW 4.5, with the libeng.dll, libmat.dll, libmx.dll ported to minGW format via pexports and dlltool.
> 
> I am posting this to get information whether somebody else met with it, or everybody else are happily using matGetVariable() on Win7 64-bit without errors on large files.

I am also using Win7 64-bit (and Matlab r2011b) and was experiencing a very similar problem.  I didn't pinpoint exactly where I could not read past, but my datafiles were ~40MB in memory (as determined by 'whos').

What fixed this issue for me was to save the file using the v7.3 Mat-file format, as opposed to the default v7.0 format.  This approximately doubled the size of the saved data file, but it fixed the read errors.  What is strange about this is that the only stated difference between v7.0 & v7.3 is the ability to read files larger than 2GB, but there must have also been some bug fixes.
0
Reply Allan 12/19/2011 5:53:08 PM

2 Replies
581 Views

(page loaded in 0.044 seconds)

Similiar Articles:









7/24/2012 1:12:48 AM


Reply: