while using parfor loop, I get "The process cannot access the file because it is being used by another process." error generated from an executable function embedded in the parfor loop. The function opens a file "tmp.key" and writes an image file "tmp.pgm" into "tmp.key" file. However, it seems that when this parfor loop runs on parallel processors, the processors are unable to access this "tmp.key" at the same time, and hence, the error is generated.
First of all, is my assesment correct? secondly, how to resolve this issue and use parfor loop successfully.

Any help is greatly appreciated, please.

Irteza 

"Syed " <irtezaa@gatech.edu> writes:

> while using parfor loop, I get "The process cannot access the file
> because it is being used by another process." error generated from an
> executable function embedded in the parfor loop. The function opens a
> file "tmp.key" and writes an image file "tmp.pgm" into "tmp.key"
> file. However, it seems that when this parfor loop runs on parallel
> processors, the processors are unable to access this "tmp.key" at the
> same time, and hence, the error is generated.  First of all, is my
> assesment correct? secondly, how to resolve this issue and use parfor
> loop successfully.
>
> Any help is greatly appreciated, please.

It sounds like your assessment is correct. By default, we keep
matlabpool workers having the same current directory as your MATLAB
client, but it sounds like you need to override this when calling your
executable. I'd proceed by making a wrapper function around the
executable which does something like this:

%---8<---8<---8<---8<---8<---
function result = wrapCallToExecutable(args)
t = getCurrentTask(); % empty on the client
if ~isempty(t)
    % Get a unique directory name and create that
    uniqueDirName = tempname();
    ok = mkdir(uniqueDirName);
    assert(ok);

    % remember the old working directory, and use onCleanup
    % to get back there    
    oldPwd = pwd;
    restorePwd = onCleanup(@() cd(oldPwd));
  
    % make an onCleanup to remove the temporary directory
    removeTempDir = onCleanup(@() rmdir(uniqueDirName, 's'));
    % move to the unique directory
    cd(uniqueDirName);
end

% call executable
system(sprintf('exeName.exe %s', args));

% do stuff, fill out 'result'.

end
%---8<---8<---8<---8<---8<---

Cheers,

Edric.

I have the same problem, but cant seem to get a solution working. My Code is

function myFunc_A()
parfor i = 1: 32
myFunc_B(i, filePaths(from:to));
end
end

function myFunc_B(i, data)
for a = 1 : A
  system('C:\unzip_from_rar_to_mat.exe'); %each file is unzipped to a unique dir.
   load(data(a)); % I fail on this call.
end
end

The error is "The process cannot access the file because it is being used by another process". 

I have the same problem, but cant seem to get a solution working. My Code is

function myFunc_A()
parfor i = 1: 32
myFunc_B(i, filePaths(from:to));
end
end

function myFunc_B(i, data)
for a = 1 : A
  system('C:\unzip_from_rar_to_mat.exe'); %each file is unzipped to a unique dir.
   load(data(a)); % I fail on this call.
end
end

The error is "The process cannot access the file because it is being used by another process". 

I use the below function to call system,

%http://uk.mathworks.com/matlabcentral/newsreader/view_thread/327867

function st_wrapCallToExecutable(args)
t = getCurrentTask(); % empty on the client
%if ~isempty(t)
    % Get a unique directory name and create that
    uniqueDirName = tempname();
    ok = mkdir(uniqueDirName);
    assert(ok);
    
    % remember the old working directory, and use onCleanup
    % to get back there
    oldPwd = pwd;
    restorePwd = onCleanup(@() cd(oldPwd));
    
    % make an onCleanup to remove the temporary directory
    removeTempDir = onCleanup(@() rmdir(uniqueDirName, 's'));
    % move to the unique directory
    cd(uniqueDirName);
%end

% call executable
%system(sprintf('exeName.exe %s', args));
system(args);
% do stuff, fill out 'result'.
end

however, this then gives a new error:

"Warning: The following error was caught while executing 'onCleanup' class destructor:
C:\Users\ADMINI~1\AppData\Local\Temp\2\tpd08fd645_a649_4bef_a97c_7a5e9dba1dd7 could not be removed."

Any suggestions?

"none " <none@gmail.com> writes:

> I use the below function to call system,
>
> %http://uk.mathworks.com/matlabcentral/newsreader/view_thread/327867
>
> function st_wrapCallToExecutable(args)
> t = getCurrentTask(); % empty on the client
> %if ~isempty(t)
>    % Get a unique directory name and create that
>    uniqueDirName = tempname();
>    ok = mkdir(uniqueDirName);
>    assert(ok);
>     % remember the old working directory, and use onCleanup
>    % to get back there
>    oldPwd = pwd;
>    restorePwd = onCleanup(@() cd(oldPwd));
>     % make an onCleanup to remove the temporary directory
>    removeTempDir = onCleanup(@() rmdir(uniqueDirName, 's'));
>    % move to the unique directory
>    cd(uniqueDirName);
> %end
>
> % call executable
> %system(sprintf('exeName.exe %s', args));
> system(args);
> % do stuff, fill out 'result'.
> end
>
> however, this then gives a new error:
>
> "Warning: The following error was caught while executing 'onCleanup' class destructor:
> C:\Users\ADMINI~1\AppData\Local\Temp\2\tpd08fd645_a649_4bef_a97c_7a5e9dba1dd7 could not be removed."
>
> Any suggestions?

That warning is because some process is still accessing the directory
that is trying to be removed. It might help to do something like this -
instead of creating the two separate onCleanups, create a single onCleanup that
does both steps (untested code!)

cleanupPwdAndTmpdir = onCleanup(@() cleanupAndRestore(uniqueDirName, oldPwd));

with

function cleanupAndRestore(dirToRemove, wdToRestore)
  cd(wdToRestore);
  t = tic;
  msg = ''; msgid = '';
  while toc(t) < 5 % try for up to 5 seconds
    [ok, msg, msgid] = rmdir(dirToRemove, 's');
    if ok
      return
    end
  end
  % if we got here, failed to remove directory
  if exist(dirToRemove, 'dir') == 7
    warning('Failed to remove directory: %s. Message: %s - %s', ...
      dirToRemove, msgid, msg);
  end
end

Cheers,
Edric.