Cisco help needed
Discussion
I'm an infrastructure/storage analyst, not network so I have half a clue about networks but probably just enough to be dangerous, so please bear with me!
I have two pairs of CBS350-48T-4X switches. Each pair is stacked, that bit seems to be working fine. Each stack is in a different comms room and I have two pairs of single-mode fibre between the rooms connected to single-mode SFPs in the SFP+ ports.
Each switch in each stack has one fibre uplink to the corresponding switch in the other stack so I've got a LAG created between the ports on each switch, for redundancy, using LACP. I have that LAG set up as a trunk on both sides with the same configuration; VLAN 1 is native/untagged, other VLANs (2 and 30) are tagged.
Devices on one stack cannot reach devices on the same VLANs/subnets on the other stack. What am I missing or just done wrong?
I have two pairs of CBS350-48T-4X switches. Each pair is stacked, that bit seems to be working fine. Each stack is in a different comms room and I have two pairs of single-mode fibre between the rooms connected to single-mode SFPs in the SFP+ ports.
Each switch in each stack has one fibre uplink to the corresponding switch in the other stack so I've got a LAG created between the ports on each switch, for redundancy, using LACP. I have that LAG set up as a trunk on both sides with the same configuration; VLAN 1 is native/untagged, other VLANs (2 and 30) are tagged.
Devices on one stack cannot reach devices on the same VLANs/subnets on the other stack. What am I missing or just done wrong?
I'm not sure what you're looking for specifically (STP/RSTP maybe?) but I've run what I can of what you asked for (truncated out ports which aren't relevant etc. for brevity):
wespahstaprdsw1#sh int status
Flow Link Back Mdix
Port Type Duplex Speed Neg ctrl State Pressure Mode
-------- ------------ ------ ----- -------- ---- ----------- -------- -------
...
te1/0/3 10G-Fiber Full 10000 Disabled Off Up Disabled Off
...
te2/0/3 10G-Fiber Full 10000 Disabled Off Up Disabled Off
...
Flow Link
Ch Type Duplex Speed Neg control State
-------- ------- ------ ----- -------- ------- -----------
Po2 10G Full 10000 Enabled Off Up
wespahstaprdsw1#show interfaces port-channel
Load balancing: src-dst-mac.
Gathering information...
Channel Ports
------- -----
Po2 Active: te1/0/3,te2/0/3
wespahstaprdsw1#show cdp neighbors
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
S - Switch, H - Host, I - IGMP, r - Repeater, P - VoIP Phone
M - Remotely-Managed Device, C - CAST Phone Port,
W - Two-Port MAC Relay
Device ID Local Adv Time To Capability Platform Port ID
Interface Ver. Live
------------------ ----------- ---- ------- ---------- ------------ -----------
ec01d56f3772 gi1/0/48 2 172 R S I Cisco gi2/0/48
CBS350-48T-4
X
(PID:CBS350-
48T-4X)-VSD
ec01d56f53b2 gi1/0/48 2 127 R S I Cisco gi1/0/48
CBS350-48T-4
X
(PID:CBS350-
48T-4X)-VSD
ec01d56f53b2 gi1/0/48 2 126 R S I Cisco gi2/0/48
CBS350-48T-4
X
(PID:CBS350-
48T-4X)-VSD
ec01d56f53b2 te1/0/3 2 124 R S I Cisco te1/0/3
CBS350-48T-4
X
(PID:CBS350-
48T-4X)-VSD
ec01d56f3772 gi2/0/48 2 169 R S I Cisco gi1/0/48
CBS350-48T-4
X
(PID:CBS350-
48T-4X)-VSD
ec01d56f53b2 gi2/0/48 2 124 R S I Cisco gi1/0/48
CBS350-48T-4
X
(PID:CBS350-
48T-4X)-VSD
ec01d56f53b2 gi2/0/48 2 124 R S I Cisco gi2/0/48
CBS350-48T-4
X
(PID:CBS350-
48T-4X)-VSD
ec01d56f53b2 te2/0/3 2 124 R S I Cisco te2/0/3
CBS350-48T-4
X
(PID:CBS350-
48T-4X)-VSD
wespahstaprdsw1#sh int switchport Po2
S-VLAN Ethernet Type: 0x8100 (802.1q)
VLAN Mapping Tunnel L2 protocols Global CoS: 5
Name: Po2
Switchport: enable
Administrative Mode: trunk
Operational Mode: up
Access Mode VLAN: 1
Access Multicast TV VLAN: none
Trunking Native Mode VLAN: 1
Trunking VLANs: 1-2,30
3-29,31-4094 (Inactive)
General PVID: 1
General VLANs: none
General Egress Tagged VLANs: none
General Forbidden VLANs: none
General Ingress Filtering: enabled
General Acceptable Frame Type: all
General GVRP status: disabled
Customer Mode VLAN: none
Customer Multicast TV VLANs: none
Private-vlan promiscuous-association primary VLAN: none
Private-vlan promiscuous-association Secondary VLANs: none
Private-vlan host-association primary VLAN: none
Private-vlan host-association Secondary VLAN: none
VLAN Mapping Tunnel - no resources
VLAN Mapping One-To-One - no resources
Classification rules:
Classification type Group ID VLAN ID
------------------- -------- -------
wespahstaprdsw1#sh run
config-file-header
<hostname>
v3.0.0.69 / RCBS3.0_930_770_008
CLI v1.0
file SSD indicator encrypted
@
ssd-control-start
ssd config
ssd file passphrase control unrestricted
no ssd file integrity control
ssd-control-end cb0a3fdb1f3a1af4e4430033719968c0
!
!
unit-type-control-start
unit-type unit 1 network gi uplink te
unit-type unit 2 network gi uplink te
unit-type unit 3 network gi uplink te
unit-type unit 4 network gi uplink te
unit-type-control-end
!
vlan database
vlan 2,30
exit
voice vlan oui-table add 0001e3 Siemens_AG_phone
voice vlan oui-table add 00036b Cisco_phone
voice vlan oui-table add 00096e Avaya
voice vlan oui-table add 000fe2 H3C_Aolynk
voice vlan oui-table add 0060b9 Philips_and_NEC_AG_phone
voice vlan oui-table add 00d01e Pingtel_phone
voice vlan oui-table add 00e075 Polycom/Veritel_phone
voice vlan oui-table add 00e0bb 3Com_phone
bonjour interface range vlan 1
hostname <hostname>
username adm password encrypted <password>
username admin password encrypted <password> privilege 15
ip ssh server
snmp-server location "<redacted>"
snmp-server contact Infrastructure
no ip http server
ip domain name <domain>
ip name-server <nameserver>
!
interface vlan 1
ip address <ipaddr> <netmask>
no ip address dhcp
!
interface vlan 2
name Private
ip address <ipaddr> <netmask>
!
interface vlan 30
name VMotion
ip address <ipaddr> <netmask>
!
interface TenGigabitEthernet1/0/1
spanning-tree link-type point-to-point
switchport mode trunk
macro description switch
!next command is internal.
macro auto smartport dynamic_type switch
!
interface TenGigabitEthernet1/0/2
spanning-tree link-type point-to-point
switchport mode trunk
macro description switch
!next command is internal.
macro auto smartport dynamic_type switch
!
interface TenGigabitEthernet1/0/3
spanning-tree link-type point-to-point
channel-group 2 mode auto
macro description switch
!next command is internal.
macro auto smartport dynamic_type switch
!
interface TenGigabitEthernet2/0/3
spanning-tree link-type point-to-point
channel-group 2 mode auto
macro description switch
!next command is internal.
macro auto smartport dynamic_type switch
!
interface Port-Channel1
description mgmt-uplink
spanning-tree link-type point-to-point
macro description switch
!next command is internal.
macro auto smartport dynamic_type switch
!
interface Port-Channel2
description xlink
spanning-tree link-type point-to-point
switchport mode trunk
macro description switch
!next command is internal.
macro auto smartport dynamic_type switch
!
exit
ip default-gateway <gateway>
I do remember something about a VLAN not coming up (can see that from show vlan) if there was not an IP addressed assigned to the VLAN. Perhaps add an IP address to the VLAN on both switches (make then different but on the same subnet).
Then should be able to ping from the switches to each other hopefully.
Then should be able to ping from the switches to each other hopefully.
Thanks. Yes there are ports on each VLAN (untagged access ports, if that matters?) and devices using them. I can ping IPs on the same stack from another device, just not IPs on the other stack, over the trunk.
Both switches look exactly the same:
I didn't configure them from the CLI, I used the web interface - in case that matters?
Both switches look exactly the same:
<stack1>#show vlan
Created by: D-Default, S-Static, G-GVRP, R-Radius Assigned VLAN, V-Voice VLAN
Vlan Name Tagged Ports UnTagged Ports Created by
---- ----------------- ------------------ ------------------ ----------------
1 1 gi1/0/25-47, DV
te1/0/1-2,te1/0/4,
gi2/0/25-47,
te2/0/1-2,te2/0/4,
gi3/0/1-48,
te3/0/1-4,
gi4/0/1-48,
te4/0/1-4,Po1-8
2 Private te1/0/1-2,Po2 gi1/0/1,gi1/0/3, S
gi1/0/5,gi1/0/7,
gi1/0/9,gi1/0/11,
gi1/0/13,gi1/0/15,
gi1/0/17,gi1/0/19,
gi1/0/21,gi1/0/23,
gi2/0/1,gi2/0/3,
gi2/0/5,gi2/0/7,
gi2/0/9,gi2/0/11,
gi2/0/13,gi2/0/15,
gi2/0/17,gi2/0/19,
gi2/0/21,gi2/0/23
30 VMotion te1/0/1-2,Po2 gi1/0/2,gi1/0/4, S
gi1/0/6,gi1/0/8,
gi1/0/10,gi1/0/12,
gi1/0/14,gi1/0/16,
gi1/0/18,gi1/0/20,
gi1/0/22,gi1/0/24,
gi2/0/2,gi2/0/4,
gi2/0/6,gi2/0/8,
gi2/0/10,gi2/0/12,
gi2/0/14,gi2/0/16,
gi2/0/18,gi2/0/20,
gi2/0/22,gi2/0/24
<stack2>#show vlan
Created by: D-Default, S-Static, G-GVRP, R-Radius Assigned VLAN, V-Voice VLAN
Vlan Name Tagged Ports UnTagged Ports Created by
---- ----------------- ------------------ ------------------ ----------------
1 1 gi1/0/25-47, DV
te1/0/1-2,te1/0/4,
gi2/0/25-47,
te2/0/1-2,te2/0/4,
gi3/0/1-48,
te3/0/1-4,
gi4/0/1-48,
te4/0/1-4,Po1-8
2 Private te1/0/1-2,Po2 gi1/0/1,gi1/0/3, S
gi1/0/5,gi1/0/7,
gi1/0/9,gi1/0/11,
gi1/0/13,gi1/0/15,
gi1/0/17,gi1/0/19,
gi1/0/21,gi1/0/23,
gi2/0/1,gi2/0/3,
gi2/0/5,gi2/0/7,
gi2/0/9,gi2/0/11,
gi2/0/13,gi2/0/15,
gi2/0/17,gi2/0/19,
gi2/0/21,gi2/0/23
30 VMotion te1/0/1-2,Po2 gi1/0/2,gi1/0/4, S
gi1/0/6,gi1/0/8,
gi1/0/10,gi1/0/12,
gi1/0/14,gi1/0/16,
gi1/0/18,gi1/0/20,
gi1/0/22,gi1/0/24,
gi2/0/2,gi2/0/4,
gi2/0/6,gi2/0/8,
gi2/0/10,gi2/0/12,
gi2/0/14,gi2/0/16,
gi2/0/18,gi2/0/20,
gi2/0/22,gi2/0/24
I didn't configure them from the CLI, I used the web interface - in case that matters?
wespahstaprdsw1#sh int status
Flow Link Back Mdix
Port Type Duplex Speed Neg ctrl State Pressure Mode
-------- ------------ ------ ----- -------- ---- ----------- -------- -------
...
te1/0/3 10G-Fiber Full 10000 Disabled Off Up Disabled Off
...
te2/0/3 10G-Fiber Full 10000 Disabled Off Up Disabled Off
...
interface TenGigabitEthernet1/0/1
spanning-tree link-type point-to-point
switchport mode trunk
macro description switch
!next command is internal.
macro auto smartport dynamic_type switch
!
interface TenGigabitEthernet1/0/2
spanning-tree link-type point-to-point
switchport mode trunk
macro description switch
!next command is internal.
macro auto smartport dynamic_type switch
!
interface TenGigabitEthernet1/0/3
spanning-tree link-type point-to-point
channel-group 2 mode auto
macro description switch
!next command is internal.
macro auto smartport dynamic_type switch
!
interface TenGigabitEthernet2/0/3
spanning-tree link-type point-to-point
channel-group 2 mode auto
macro description switch
!next command is internal.
macro auto smartport dynamic_type switch
Just an idea (last time I looked at this it was Fa0 and Gi0 was revolutionary!) shouldn't te1/0/3 and te2/0/3 be switchport mode trunk if that's where the fibres are?
These switches have odd CLI but i digress,
your ports you're 'trunking' te1/0/3 and 2/0/3 they look like they're setup as switchports and no vlan defined so presumably vlan1 there's no command 'switchport mode trunk' on either interface
what's the config when you run
show run int po2
should have the trunk config in there too, though if you apply to the logical port channel it'll push to the physical interfaces
your ports you're 'trunking' te1/0/3 and 2/0/3 they look like they're setup as switchports and no vlan defined so presumably vlan1 there's no command 'switchport mode trunk' on either interface
what's the config when you run
show run int po2
should have the trunk config in there too, though if you apply to the logical port channel it'll push to the physical interfaces
Edited by Richyvrlimited on Wednesday 8th February 14:25
8bit said:
Thanks. Yes there are ports on each VLAN (untagged access ports, if that matters?) and devices using them. I can ping IPs on the same stack from another device, just not IPs on the other stack, over the trunk.
I didn't configure them from the CLI, I used the web interface - in case that matters?
No problem using the GUI - it will just create the config for you.I didn't configure them from the CLI, I used the web interface - in case that matters?
If you can ping from machines on the same VLAN on the same switch then it means that the trunk is not allowing VLAN traffic to flow between the switches - show interfaces trunk should give you and idea what's happening.
Are te1/0/3 and te2/0/3 are the ports with the SM fibre links? If you are using two ports and trying to portchannel them together to give 20G then maybe just simplify with a single 10G connection to get that part working (can always add the other in when proved).
To get a trunk between switches should only require something like:
switchport mode trunk
switchport trunk encapsulation dot1q
with no other config needed.
Also, try the assigning IP address to the VLAN on both switches trick as I do think that might help bring the trunk up.
To get a trunk between switches should only require something like:
switchport mode trunk
switchport trunk encapsulation dot1q
with no other config needed.
Also, try the assigning IP address to the VLAN on both switches trick as I do think that might help bring the trunk up.
pistonheadforum said:
Are te1/0/3 and te2/0/3 are the ports with the SM fibre links? If you are using two ports and trying to portchannel them together to give 20G then maybe just simplify with a single 10G connection to get that part working (can always add the other in when proved).
To get a trunk between switches should only require something like:
switchport mode trunk
switchport trunk encapsulation dot1q
with no other config needed.
Also, try the assigning IP address to the VLAN on both switches trick as I do think that might help bring the trunk up.
FYI the encapsulation command will error on newer switches, it's only needed on older boxes from before dot1q tagging became ubiquitous over ISLTo get a trunk between switches should only require something like:
switchport mode trunk
switchport trunk encapsulation dot1q
with no other config needed.
Also, try the assigning IP address to the VLAN on both switches trick as I do think that might help bring the trunk up.
no need to create an SVI on both switches, just create the L2 vlan and as long as it's assigned to an access or trunk port the vlan will come up.
if there's an SVI with no IP address that should be deleted, it's not needed
pistonheadforum said:
No problem using the GUI - it will just create the config for you.
If you can ping from machines on the same VLAN on the same switch then it means that the trunk is not allowing VLAN traffic to flow between the switches - show interfaces trunk should give you and idea what's happening.
show int trunk gives me "bad parameter value". Did you mean port-channel instead of trunk, cos that works:If you can ping from machines on the same VLAN on the same switch then it means that the trunk is not allowing VLAN traffic to flow between the switches - show interfaces trunk should give you and idea what's happening.
<switchname>#show interfaces port-channel
Load balancing src-dst-mac.
Gathering information...
Channel Ports
------- -----
Po1 Active: gi1/0/48,gi2/0/48
Po2 Active: te1/0/3,te2/0/3
Richyvrlimited said:
These switches have odd CLI but i digress,
your ports you're 'trunking' te1/0/3 and 2/0/3 they look like they're setup as switchports and no vlan defined so presumably vlan1 there's no command 'switchport mode trunk' on either interface
what's the config when you run
show run int po2
should have the trunk config in there too, though if you apply to the logical port channel it'll push to the physical interfaces
Here's the running config for po2 and each of the physical interfaces:your ports you're 'trunking' te1/0/3 and 2/0/3 they look like they're setup as switchports and no vlan defined so presumably vlan1 there's no command 'switchport mode trunk' on either interface
what's the config when you run
show run int po2
should have the trunk config in there too, though if you apply to the logical port channel it'll push to the physical interfaces
Edited by Richyvrlimited on Wednesday 8th February 14:25
wespahstaprdsw1#show run int po2
interface Port-Channel2
description xlink
spanning-tree link-type point-to-point
switchport mode trunk
macro description switch
!next command is internal.
macro auto smartport dynamic_type switch
!
wespahstaprdsw1#show run int te1/0/3
interface TenGigabitEthernet1/0/3
spanning-tree link-type point-to-point
channel-group 2 mode auto
macro description switch
!next command is internal.
macro auto smartport dynamic_type switch
!
wespahstaprdsw1#show run int te2/0/3
interface TenGigabitEthernet2/0/3
spanning-tree link-type point-to-point
channel-group 2 mode auto
macro description switch
!next command is internal.
macro auto smartport dynamic_type switch
!
essayer said:
Just an idea (last time I looked at this it was Fa0 and Gi0 was revolutionary!) shouldn't te1/0/3 and te2/0/3 be switchport mode trunk if that's where the fibres are?
I'm not sure - I'm not using the switch ports directly, they're the two members of a LAG, would I need the member ports configured as trunk ports as well as the overall LAG for this to work?pistonheadforum said:
Are te1/0/3 and te2/0/3 are the ports with the SM fibre links? If you are using two ports and trying to portchannel them together to give 20G then maybe just simplify with a single 10G connection to get that part working (can always add the other in when proved).
To get a trunk between switches should only require something like:
switchport mode trunk
switchport trunk encapsulation dot1q
with no other config needed.
Also, try the assigning IP address to the VLAN on both switches trick as I do think that might help bring the trunk up.
I did originally start with just standalone 10GbE interfaces connected together and set those up as trunks, that didn't work either. Not sure what you mean by the IP address assignment trick? Each stack has an IP address in all three VLANs' subnets. The first VLAN (ID 1) is the subnet they connect to the customer site switches with, each stack can ping the other but it's not clear whether that's traversing my trunk or over the customer network. Neither stack can ping the other on VLANs 2 or 30 though. Those VLANs are not on the client network and the ports they connect to client network with are not trunked either.To get a trunk between switches should only require something like:
switchport mode trunk
switchport trunk encapsulation dot1q
with no other config needed.
Also, try the assigning IP address to the VLAN on both switches trick as I do think that might help bring the trunk up.
8bit said:
I did originally start with just standalone 10GbE interfaces connected together and set those up as trunks, that didn't work either. Not sure what you mean by the IP address assignment trick? Each stack has an IP address in all three VLANs' subnets. The first VLAN (ID 1) is the subnet they connect to the customer site switches with, each stack can ping the other but it's not clear whether that's traversing my trunk or over the customer network. Neither stack can ping the other on VLANs 2 or 30 though. Those VLANs are not on the client network and the ports they connect to client network with are not trunked either.
If the switches can ping each other without using your link then it sounds like they are already connected (perhaps through the customers network?). If they are connected already then spanning tree might be spotting a loop and keeping your trunk interface down to prevent loops in the network. It can say it's up, but really it's down waiting for the spanning-tree keep-alives to tell it it's safe to bring it back up.I think if VLAN 1 is routable but the others arent then it might be that the link is not coming up. You might be able to prevent VLAN1 going over your link by limiting the VLANs on your trunk:
switchport trunk allowed vlan 2, 3
Can you draw out what your network looks like?
Edited by pistonheadforum on Wednesday 8th February 15:59
pistonheadforum said:
If the switches can ping each other without using your link then it sounds like they are already connected (perhaps through the customers network?). If they are connected already then spanning tree might be spotting a loop and keeping your trunk interface down to prevent loops in the network. It can say it's up, but really it's down waiting for the spanning-tree keep-alives to tell it it's safe to bring it back up.
Can you draw out what your network looks like?
They can ping but it's not clear to me whether they're doing so across the trunk or via the customer network. I don't have physical access to either stack just now to pull one out of the customer network to test. Doing a traceroute from the web UI of either switch to the other doesn't show any hops along the way, hop 1 is just the other stack. They're on the same subnet though so not sure if that confirms which path they're taking.Can you draw out what your network looks like?
Excuse the crappy drawing (even worse with that than with networks) but this is basically what it looks like. The two horizontal uplinks make up the LAG I'm trying to run the trunk over.
Gassing Station | Computers, Gadgets & Stuff | Top of Page | What's New | My Stuff