Let's say, for example, I would like to read a file called 'data.dat',
which has these three real numbers:
1.000000
2.000000
3.000000
And.. I have been trying to read these three number with mpi_file_read
by following this sample code.
program test5
include 'mpif.h'
integer bufsize, numread, totprocessed, status(MPI_STATUS_SIZE)
parameter (bufsize=3)
real localbuffer(bufsize)
call mpi_init(ierr)
call mpi_comm_size(MPI_COMM_WORLD, num_nodes, ierr)
call mpi_comm_rank(MPI_COMM_WORLD, iam, ierr)
call MPI_FILE_OPEN( MPI_COMM_WORLD, 'data',
& MPI_MODE_RDONLY, MPI_INFO_NULL, myfh, ierr )
call MPI_FILE_SET_VIEW( myfh, 0, MPI_REAL, MPI_REAL, 'native',
& MPI_INFO_NULL, ierr )
totprocessed = 0
do
call MPI_FILE_READ( myfh, localbuffer, bufsize, MPI_REAL,
& status, ierr )
write(*,*) localbuffer
call MPI_GET_COUNT( status, MPI_REAL, numread, ierr )
! call process_input( localbuffer, numread )
totprocessed = totprocessed + numread
if ( numread < bufsize ) exit
enddo
write(6,1001) numread, bufsize, totprocessed
1001 format( "No more data: read", I3, "and expected", I3,
& "Processed total of", I6, "before terminating job." )
call MPI_FILE_CLOSE( myfh, ierr )
call mpi_finalize(ierr)
stop
end
And if I write out 'localbuffer' to screen it only writes out three
zeros(not assigned numbers) instead of the numbers in the file. I
don't think that MPI_FILE_READ routine actually read anything.
It seems like I have no idea what goes on. Can anyone explain to me
how to read a file using mpi_file_read?
|
|
0
|
|
|
|
Reply
|
syraero (5)
|
1/10/2006 9:43:18 PM |
|
syraero@gmail.com wrote:
[...]
> And if I write out 'localbuffer' to screen it only writes out three
> zeros(not assigned numbers) instead of the numbers in the file. I
> don't think that MPI_FILE_READ routine actually read anything.
> It seems like I have no idea what goes on. Can anyone explain to me
> how to read a file using mpi_file_read?
You should *always* check the error codes of all MPI-IO functions that you call.
From your description, it's likely that MPI_FILE_OPEN fails because it tries to
open the file 'data', while you talk about a file called 'data.dat'.
--
Joachim - reply to joachim at domain ccrl-nece dot de
Opinion expressed is personal and does not constitute
an opinion or statement of NEC Laboratories.
|
|
0
|
|
|
|
Reply
|
Joachim
|
1/11/2006 9:49:28 AM
|
|
>Let's say, for example, I would like to read a file called 'data.dat',
> call MPI_FILE_OPEN( MPI_COMM_WORLD, 'data',
> & MPI_MODE_RDONLY, MPI_INFO_NULL, myfh, ierr )
As Joachim noted, these names have to agree. You are not under DOS or
some like, so the extension (.dat) isn't really a real extension; it is
just part of the name. So put 'data.dat' into the MPI_FILE_OPEN and it
should work.
I also use (.dat) in filenames but that's just to help me recognise them
when I list the directory.
--
ciao,
Bruce
drift wave turbulence: http://www.rzg.mpg.de/~bds/
|
|
0
|
|
|
|
Reply
|
Bruce
|
1/11/2006 12:27:22 PM
|
|
Sorry for the confusions. After I failed with 'data.dat', I changed
the fiename to "data", hoping it might work. So.. it wan't the problem
of extension.
I cheked the error code.
ierr from open outputs 0 and
ierr from read outpus 872468256.
myfh from both open and read outputs 135656648.
Is this supposed to mean something?
|
|
0
|
|
|
|
Reply
|
syraero
|
1/11/2006 3:58:35 PM
|
|
The other thing which might be a problem is precision... are you on a
64 bit machine? What platform are you on? Sometimes MPI_REAL has to be
MPI_DOUBLE_PRECISION
Try it and see
--
ciao,
Bruce
drift wave turbulence: http://www.rzg.mpg.de/~bds/
|
|
0
|
|
|
|
Reply
|
Bruce
|
1/12/2006 1:08:31 PM
|
|
syraero@gmail.com wrote:
> Sorry for the confusions. After I failed with 'data.dat', I changed
> the fiename to "data", hoping it might work. So.. it wan't the problem
> of extension.
>
> I cheked the error code.
> ierr from open outputs 0 and
> ierr from read outpus 872468256.
This means something went wrong with MPI_File_read(). You can translate the
error code into a readable error message using MPI_ERROR_STRING().
Next to Bruce' recommendation, you could try to omitt MPI_FILE_SET_VIEW() (it is
not needed in this specific case). In any case, check the error code!
Which platform are you on, and which MPI are you using?
--
Joachim - reply to joachim at domain ccrl-nece dot de
Opinion expressed is personal and does not constitute
an opinion or statement of NEC Laboratories.
|
|
0
|
|
|
|
Reply
|
Joachim
|
1/12/2006 1:56:56 PM
|
|
After ommited MPI_FILE_SET_VIEW(), I am getting the followings
ierr myfh
open 0 135656648
read 0 135656648
4.0273673E-11 1.4923724E-19 1.1699684E-19
Instead of three zeros as before. Still not right.
I am currently using 32bit desktop redhat linux and intel fortran
compiler and MPICH-2.
|
|
0
|
|
|
|
Reply
|
syraero
|
1/13/2006 3:10:50 PM
|
|
syraero@gmail.com wrote:
> After ommited MPI_FILE_SET_VIEW(), I am getting the followings
>
> ierr myfh
> open 0 135656648
> read 0 135656648
> 4.0273673E-11 1.4923724E-19 1.1699684E-19
>
> Instead of three zeros as before. Still not right.
How did you create the input file - does it contain binary or ASCII data?
The fact that MPI_FILE_VIEW influences the outcome of MPI_FILE_READ is strange,
though. As you use MPICH2, you could also post your problem to the MPICH2
mailing list after having verified everything else.
BTW, it would be nice if you posted non-anonymously.
--
Joachim - reply to joachim at domain ccrl-nece dot de
Opinion expressed is personal and does not constitute
an opinion or statement of NEC Laboratories.
|
|
0
|
|
|
|
Reply
|
Joachim
|
1/13/2006 4:39:54 PM
|
|
After your suggestion, I varied the file format for 'data.dat'.
formatted, unformatted, binary
Now it works. However, it only works with 'binary format' and after
omitting MPI_FILE_SET_VIEW().
Have I done something wrong? or is it a compatibility issue of intel
fortran and MPICH2?
BTW, I really appreciate you guys help.
-Jin Lee-
|
|
0
|
|
|
|
Reply
|
syraero
|
1/13/2006 8:24:01 PM
|
|
syraero@gmail.com wrote:
> After your suggestion, I varied the file format for 'data.dat'.
> formatted, unformatted, binary
> Now it works. However, it only works with 'binary format' and after
> omitting MPI_FILE_SET_VIEW().
The fact that it only works with binary format is fine: MPI-IO does not perform
data-tranformation if you use the default "native" data format; instead, it
directly reads the file data to memory. This requires the correct binary
representation in the file.
> Have I done something wrong? or is it a compatibility issue of intel
> fortran and MPICH2?
The problem with MPI_FILE_SET_VIEW() might be related to the integer length of
the 'disp' argument: it needs to be INTEGER(KIND=MPI_OFFSET_KIND), which is
8byte instead of 4byte. Maybe the compiler gets this wrong. You should take a
closer look at the error code using MPI_ERROR_STRING(), and try to use a
variable of the required type instead of a constant.
Otherwise, it might be a bug, although this is unlikely because for a contiguous
file view like yours, MPI_FILE_SET_VIEW() does not need to do very complicated
things.
--
Joachim - reply to joachim at domain ccrl-nece dot de
Opinion expressed is personal and does not constitute
an opinion or statement of NEC Laboratories.
|
|
0
|
|
|
|
Reply
|
Joachim
|
1/16/2006 8:40:47 AM
|
|
|
9 Replies
531 Views
(page loaded in 0.115 seconds)
Similiar Articles: Reading mat file in C++ (VS) - comp.soft-sys.matlabHello, I am trying to read a 2D array ("aper") from a mat file in C++ (Visual Studio) and assign it to a 2D array ("aper1"). I am using the followi... How to read/write .mat files in VB.net... - comp.soft-sys.matlab ...Hello, Like the subject says, I am trying to read and write .mat files from VB.net. Currently I am interfacing w/ MatLab and sending it open comma... Gml to Graphs - comp.soft-sys.matlabHi all: I am trying to read the gml file in matlab so that it can be converted into adjacency matrix representation of graphs. Can anybody help me in... Reading in pre-fixed file layout - comp.soft-sys.sasHello: I have a .txt file that has been zipped that I am trying to read into SAS.... The layout is this: File Layout Length || From || To || Da... textscan error: out of memory - comp.soft-sys.matlabOn 10-08-26 04:03 PM, Li wrote: > I am trying to use textscan to read data from a huge text file. Below is > the code: > > A = textscan(fid, '%f', 'delimiter ... reading text files - comp.soft-sys.sasMy text file contains a single number and I have a lot of files like that. I am trying to create a macro to read all the files and use it. Pleas let me know if there is ... Speed-up the reading of large binary files with complex structures ...Hi, I am trying to read a large binary file where the data is structured in a known format composed by a variable number of blocks. Also, these blo... reading xls without proc import or changing file type - comp.soft ...I am trying to figure out a way to read an ods generated xls file from sas that has been manipulated back into sas without using a proc import or ch... How to read files from multiple folders - comp.soft-sys.matlab ...How to read files from multiple folders - comp.soft-sys.matlab ... read multiple text files to one file - comp.soft-sys.matlab ... Hello, I am trying to read multiple text ... using awk to modify xml file - comp.lang.awkRead file twice and modify content - comp.lang.awk... to modify content of multiple files. ... Split file using awk - The UNIX and Linux Forums I am trying to read a ... Reading, Opening Excel File in C# Using Microsoft.Office.Introp ...Here i am trying to help you to how to open and read the EXCEL File and bind the data to gridview server control in asp.net using C# language. Its very easy way ... Paint cannot read this file type error message - Microsoft Answers... and using paint for years and now only recently am ... Windows does not natively read PDF files so you have to have ... I'm working on trying them. When in Default Programs ... 7/22/2012 3:10:11 AM
|