Hey all. Another somewhat off-topic one.

We're mucking around with switching ports and paths to our disk array.
The problem is that we can't get rid of Veritas DMP paths that are disabled.
With the (disabled) paths being held by Veritas, we can't use cfgadm to
clear the paths from an OS level.

We're running Solaris 8 with Veritas VxVM 4.1. MPXIO is disabled, so
everything is being run through vxdmp. Here's the DMP info from "vxdisk
list <device>" for one LUN:

Multipathing information:
numpaths:   4
c3t500060E8027A8215d162s2       state=disabled
c2t500060E8027A8205d162s2       state=enabled
c2t500060E8027A820Dd0s2 state=enabled
c3t500060E8027A821Dd0s2 state=enabled

We want to get rid of the first path, and then we're going to disable and
delete the second one as well.

vxdctl enable doesn't work. Neither does vxdisk scandisks, despite the
fact that it's supposed to force a DMP reconfiguration.

Not sure how we can clear this without unconfiguring the entire controller.
Any guesses?

Thanks,
Colin

Colin B. <cbigam@somewhereelse.nucleus.com> wrote:
> Hey all. Another somewhat off-topic one.

> We're mucking around with switching ports and paths to our disk array.
> The problem is that we can't get rid of Veritas DMP paths that are
> disabled.

How are you trying to do that?

> Multipathing information:
> numpaths:   4
> c3t500060E8027A8215d162s2       state=disabled
> c2t500060E8027A8205d162s2       state=enabled
> c2t500060E8027A820Dd0s2 state=enabled
> c3t500060E8027A821Dd0s2 state=enabled

> We want to get rid of the first path, and then we're going to disable and
> delete the second one as well.

> vxdctl enable doesn't work. Neither does vxdisk scandisks, despite the
> fact that it's supposed to force a DMP reconfiguration.

> Not sure how we can clear this without unconfiguring the entire controller.
> Any guesses?

I'd start with vxdmpadm disable xxxx.  Have you used that?

-- 
Darren Dunham                                           ddunham@taos.com
Senior Technical Consultant         TAOS            http://www.taos.com/
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >

Darren Dunham <ddunham@redwood.taos.com> wrote:
> Colin B. <cbigam@somewhereelse.nucleus.com> wrote:
>> Hey all. Another somewhat off-topic one.
> 
>> We're mucking around with switching ports and paths to our disk array.
>> The problem is that we can't get rid of Veritas DMP paths that are
>> disabled.
> 
> How are you trying to do that?

Basically, we've attached new cables from the disk (HDS9960) to the switches
(Brocade 3800), rezoned them, and are now adding new paths from the LUNs to
the host, via the new Hitachi port. Then we take away the old path.

>> Multipathing information:
>> numpaths:   4
>> c3t500060E8027A8215d162s2       state=disabled
>> c2t500060E8027A8205d162s2       state=enabled
>> c2t500060E8027A820Dd0s2 state=enabled
>> c3t500060E8027A821Dd0s2 state=enabled
> 
>> We want to get rid of the first path, and then we're going to disable and
>> delete the second one as well.
> 
>> vxdctl enable doesn't work. Neither does vxdisk scandisks, despite the
>> fact that it's supposed to force a DMP reconfiguration.
> 
>> Not sure how we can clear this without unconfiguring the entire controller.
>> Any guesses?
> 
> I'd start with vxdmpadm disable xxxx.  Have you used that?

I can do that, on an available path (i.e. the second one), and it returns
as 'state=disabled,' same as if I physically take the path away.

So some further information here. The device in question was listed as
"failing" in cfgadm -al -o show_FCP_dev. The only way I know of getting
that unstuck is to unconfigure and reconfigure the controller, so that we
did. It's now clear of format, and the device paths have been removed.
Veritas, however, now shows it as NONAME. See the following:

Multipathing information:
numpaths:   4
NONAME          state=disabled
c2t500060E8027A8205d162s2       state=disabled
c2t500060E8027A820Dd0s2 state=enabled
c3t500060E8027A821Dd0s2 state=enabled

In fact, vxdmpadm getsubpaths shows a large number of NONAME/disabled
paths, presumably one per device that's been removed since the last
reboot.

There must be a non-reboot way of eliminating all of these things. Any
further thoughts?

Thanks,
Colin

Here's an interesting update.

Colin B. <cbigam@somewhereelse.nucleus.com> wrote:
> Darren Dunham <ddunham@redwood.taos.com> wrote:
>> Colin B. <cbigam@somewhereelse.nucleus.com> wrote:
>>> Hey all. Another somewhat off-topic one.
>> 
>>> We're mucking around with switching ports and paths to our disk array.
>>> The problem is that we can't get rid of Veritas DMP paths that are
>>> disabled.
>> 
>> How are you trying to do that?
> 
> Basically, we've attached new cables from the disk (HDS9960) to the switches
> (Brocade 3800), rezoned them, and are now adding new paths from the LUNs to
> the host, via the new Hitachi port. Then we take away the old path.

(snipage)

Interestingly, if we disable the second original path to the disk, the
disk goes catatonic, and the diskgroup becomes disabled. Not good!
vxdisk list <device> shows:

Multipathing information:
numpaths:   4
NONAME          state=disabled
NONAME          state=disabled
c2t500060E8027A820Dd0s2 state=enabled
c3t500060E8027A821Dd0s2 state=enabled

So in theory, there are two accessible paths to the LUN, but the system
can't use them. Is there a limit to the number of paths that Veritas
will use?

At any rate, a reboot cleared things up nicely. I'm just really not happy
with that as a solution.

Colin

Colin B. <cbigam@somewhereelse.nucleus.com> wrote:
> Interestingly, if we disable the second original path to the disk, the
> disk goes catatonic, and the diskgroup becomes disabled. Not good!
> vxdisk list <device> shows:

> Multipathing information:
> numpaths:   4
> NONAME          state=disabled
> NONAME          state=disabled
> c2t500060E8027A820Dd0s2 state=enabled
> c3t500060E8027A821Dd0s2 state=enabled

> So in theory, there are two accessible paths to the LUN, but the system
> can't use them. Is there a limit to the number of paths that Veritas
> will use?

> At any rate, a reboot cleared things up nicely. I'm just really not happy
> with that as a solution.

I've not tried to do what you're doing and have not really seen that
behavior.  You might bring the details over to the list at
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-vx/

There's some more VxVM folks there and occasional project folks that
might know more about it.

-- 
Darren Dunham                                           ddunham@taos.com
Senior Technical Consultant         TAOS            http://www.taos.com/
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >