I am trying read a file with mpi_file_read

  • Follow


Let's say, for example, I would like to read a file called 'data.dat',
which has these three real numbers:
1.000000
2.000000
3.000000

And..  I have been trying to read these three number with mpi_file_read
by following this sample code.

      program test5
      include 'mpif.h'
      integer   bufsize, numread, totprocessed, status(MPI_STATUS_SIZE)
      parameter (bufsize=3)
      real      localbuffer(bufsize)

      call mpi_init(ierr)
      call mpi_comm_size(MPI_COMM_WORLD, num_nodes, ierr)
      call mpi_comm_rank(MPI_COMM_WORLD, iam, ierr)

      call MPI_FILE_OPEN( MPI_COMM_WORLD, 'data',
     &                   MPI_MODE_RDONLY, MPI_INFO_NULL, myfh, ierr )
      call MPI_FILE_SET_VIEW( myfh, 0, MPI_REAL, MPI_REAL, 'native',
     &                      MPI_INFO_NULL, ierr )
      totprocessed = 0
      do
         call MPI_FILE_READ( myfh, localbuffer, bufsize, MPI_REAL,
     &                      status, ierr )
         write(*,*) localbuffer
         call MPI_GET_COUNT( status, MPI_REAL, numread, ierr )
!         call process_input( localbuffer, numread )
         totprocessed = totprocessed + numread
         if ( numread < bufsize ) exit
      enddo

      write(6,1001) numread, bufsize, totprocessed
1001  format( "No more data:  read", I3, "and expected", I3,
     & "Processed total of", I6, "before terminating job." )

      call MPI_FILE_CLOSE( myfh, ierr )

      call mpi_finalize(ierr)
      stop
      end

And if I write out 'localbuffer' to screen it only writes out three
zeros(not assigned numbers) instead of the numbers in the file.  I
don't think that MPI_FILE_READ routine actually read anything.
It seems like I have no idea what goes on.  Can anyone explain to me
how to read a file using mpi_file_read?

0
Reply syraero (5) 1/10/2006 9:43:18 PM

syraero@gmail.com wrote:
[...]
> And if I write out 'localbuffer' to screen it only writes out three
> zeros(not assigned numbers) instead of the numbers in the file.  I
> don't think that MPI_FILE_READ routine actually read anything.
> It seems like I have no idea what goes on.  Can anyone explain to me
> how to read a file using mpi_file_read?

You should *always* check the error codes of all MPI-IO functions that you call. 
 From your description, it's likely that MPI_FILE_OPEN fails because it tries to 
open the file 'data', while you talk about a file called 'data.dat'.

-- 
Joachim - reply to joachim at domain ccrl-nece dot de

Opinion expressed is personal and does not constitute
an opinion or statement of NEC Laboratories.
0
Reply Joachim 1/11/2006 9:49:28 AM


>Let's say, for example, I would like to read a file called 'data.dat',

>      call MPI_FILE_OPEN( MPI_COMM_WORLD, 'data',
>     &                   MPI_MODE_RDONLY, MPI_INFO_NULL, myfh, ierr )

As Joachim noted, these names have to agree.  You are not under DOS or
some like, so the extension (.dat) isn't really a real extension; it is
just part of the name.  So put 'data.dat' into the MPI_FILE_OPEN and it
should work.

I also use (.dat) in filenames but that's just to help me recognise them
when I list the directory.

-- 
ciao,
Bruce

drift wave turbulence:  http://www.rzg.mpg.de/~bds/

0
Reply Bruce 1/11/2006 12:27:22 PM

Sorry for the confusions.  After I failed with 'data.dat', I changed
the fiename to "data", hoping it might work.  So.. it wan't the problem
of extension.

I cheked the error code.
ierr from open outputs 0   and
ierr from read outpus 872468256.

myfh from both open and read outputs 135656648.

Is this supposed to mean something?

0
Reply syraero 1/11/2006 3:58:35 PM

The other thing which might be a problem is precision...  are you on a
64 bit machine?  What platform are you on?  Sometimes MPI_REAL has to be
MPI_DOUBLE_PRECISION

Try it and see

-- 
ciao,
Bruce

drift wave turbulence:  http://www.rzg.mpg.de/~bds/

0
Reply Bruce 1/12/2006 1:08:31 PM

syraero@gmail.com wrote:
> Sorry for the confusions.  After I failed with 'data.dat', I changed
> the fiename to "data", hoping it might work.  So.. it wan't the problem
> of extension.
> 
> I cheked the error code.
> ierr from open outputs 0   and
> ierr from read outpus 872468256.

This means something went wrong with MPI_File_read(). You can translate the 
error code into a readable error message using MPI_ERROR_STRING().

Next to Bruce' recommendation, you could try to omitt MPI_FILE_SET_VIEW() (it is 
not needed in this specific case). In any case, check the error code!

Which platform are you on, and which MPI are you using?

-- 
Joachim - reply to joachim at domain ccrl-nece dot de

Opinion expressed is personal and does not constitute
an opinion or statement of NEC Laboratories.
0
Reply Joachim 1/12/2006 1:56:56 PM

After ommited MPI_FILE_SET_VIEW(), I am getting the followings

               ierr          myfh
 open          0     135656648
 read           0     135656648
  4.0273673E-11  1.4923724E-19  1.1699684E-19

Instead of three zeros as before.  Still not right.

I am currently using 32bit desktop redhat linux and intel fortran
compiler and MPICH-2.

0
Reply syraero 1/13/2006 3:10:50 PM

syraero@gmail.com wrote:
> After ommited MPI_FILE_SET_VIEW(), I am getting the followings
> 
>                ierr          myfh
>  open          0     135656648
>  read           0     135656648
>   4.0273673E-11  1.4923724E-19  1.1699684E-19
> 
> Instead of three zeros as before.  Still not right.

How did you create the input file - does it contain binary or ASCII data?

The fact that MPI_FILE_VIEW influences the outcome of MPI_FILE_READ is strange, 
though. As you use MPICH2, you could also post your problem to the MPICH2 
mailing list after having verified everything else.

BTW, it would be nice if you posted non-anonymously.

-- 
Joachim - reply to joachim at domain ccrl-nece dot de

Opinion expressed is personal and does not constitute
an opinion or statement of NEC Laboratories.
0
Reply Joachim 1/13/2006 4:39:54 PM

After your suggestion, I varied the file format for 'data.dat'.
formatted, unformatted, binary
Now it works.  However, it only works with 'binary format' and after
omitting MPI_FILE_SET_VIEW().
Have I done something wrong?  or is it a compatibility issue of intel
fortran and MPICH2?

BTW, I really appreciate you guys help.

-Jin Lee-

0
Reply syraero 1/13/2006 8:24:01 PM

syraero@gmail.com wrote:
> After your suggestion, I varied the file format for 'data.dat'.
> formatted, unformatted, binary
> Now it works.  However, it only works with 'binary format' and after
> omitting MPI_FILE_SET_VIEW().

The fact that it only works with binary format is fine: MPI-IO does not perform 
data-tranformation if you use the default "native" data format; instead, it 
directly reads the file data to memory. This requires the correct binary 
representation in the file.

> Have I done something wrong?  or is it a compatibility issue of intel
> fortran and MPICH2?

The problem with MPI_FILE_SET_VIEW() might be related to the integer length of 
the 'disp' argument: it needs to be INTEGER(KIND=MPI_OFFSET_KIND), which is 
8byte instead of 4byte. Maybe the compiler gets this wrong. You should take a 
closer look at the error code using MPI_ERROR_STRING(), and try to use a 
variable of the required type instead of a constant.

Otherwise, it might be a bug, although this is unlikely because for a contiguous 
file view like yours, MPI_FILE_SET_VIEW() does not need to do very complicated 
things.

-- 
Joachim - reply to joachim at domain ccrl-nece dot de

Opinion expressed is personal and does not constitute
an opinion or statement of NEC Laboratories.
0
Reply Joachim 1/16/2006 8:40:47 AM

9 Replies
531 Views

(page loaded in 0.115 seconds)

Similiar Articles:













7/22/2012 3:10:11 AM


Reply: