PCI Surgery with /etc/path_to_inst
How to renumber PCI devices in a Solaris environment by removing stale hardware references, such as those left behind after replacing a network card.
Introduction
We recently came across a Solaris 8 x86 system that was not listing its network cards correctly in the output of netstat -i. Here's how we fixed it:
Background Information
Originally we were trying to use the snoop utility, and being greeted with the rather unhelpful error message:
bash-2.05# snoop dlattachreq: DL_ERROR_ACK: dl_errno 8 unix_errno 0
On further investigation, the machine's network card was not shown in netstat -i:
bash-2.05# netstat -in Name Mtu Net/Dest Address Ipkts Ierrs Opkts Oerrs Collis Queue lo0 8232 127.0.0.0 127.0.0.1 552 0 552 0 0 0
However, netstat -ian showed the card as a virtual interface rather than a physical one:
bash-2.05# netstat -ian Name Mtu Net/Dest Address Ipkts Ierrs Opkts Oerrs Collis Queue lo0 8232 127.0.0.0 127.0.0.1 552 0 552 0 0 0 lo0 8232 127.0.0.0 127.0.0.1 0 N/A 536 N/A N/A 0 iprb0 1500 10.10.10.0 10.10.10.10 6711 N/A 8155 N/A N/A 0
Looking Further
bash-2.05# ls -al /dev/iprb* lrwxrwxrwx 1 root root 35 May 25 2001 /dev/iprb -> ../devices/pci@0,0/pci8086,1@b:iprb lrwxrwxrwx 1 root root 36 May 25 2001 /dev/iprb0 -> ../devices/pci@0,0/pci8086,1@b:iprb1 lrwxrwxrwx 1 root root 36 May 25 2001 /dev/iprb1 -> ../devices/pci@0,0/pci8086,1@b:iprb1
Here, /dev/iprb0 is the same device as /dev/iprb1. We then examine the /devices tree for iprb information:
bash-2.05# ls -l /devices/pci@0,0/*iprb crw------- 1 root sys 27, 0 Apr 24 2001 /devices/pci@0,0/pci8086,1@9:iprb crw------- 1 root sys 27, 0 May 25 2001 /devices/pci@0,0/pci8086,1@b:iprb
It appears this machine had its NIC changed at some point and is still referencing it somewhere. We check the /etc/path_to_inst file:
bash-2.05# grep iprb /etc/path_to_inst "/pci@0,0/pci8086,1@9" 0 "iprb" "/pci@0,0/pci8086,1@b" 1 "iprb"
Removing the Erroneous Reference
Firstly, we edit the above file so that the only remaining iprb line is:
"/pci@0,0/pci8086,1@b" 0 "iprb"
We then delete the appropriate device files:
bash-2.05# rm /dev/iprb* bash-2.05# cd /devices/pci@0,0/ bash-2.05# rm pci8086,1@9:iprb pci8086,1@b:iprb pci8086,1@b:iprb1
Finally, we perform a reconfiguration reboot:
bash-2.05# reboot -- -r
The machine reboots, reconfigures, and comes back up with a single network interface.
Conclusion
To solve PCI numbering problems, or to remove stale hardware references, the /etc/path_to_inst file is your friend. Please note, however, that corrupting this file can stop your machine from booting properly.
In that case, a boot -a must be performed and the file recreated from scratch. This has the side-effect of renumbering all PCI devices, so disk controllers and other hardware may also move.
John Cartwright <johnc@grok.org.uk>Editor's Note: 2002-03-23
Note that this issue is not specific to Solaris 8 x86; similar behaviour has also been observed on Solaris 7.