We have a LAN with 2x Cisco 4500's as gateways running HSRP. We're using Exterity HD IP Encoders to take HD video and put it onto the network as a multicast UDP stream (playable in VLC).
I have a fairely extensive Nagios setup on Linux and would like to find some way to check that:
- Multicast stream is on the network.
- Multicast stream isn't frozen, so check for audio or ...
- Confirm the source IP of the stream matches what we expect from the multicast address.
1 and 3 could be combined maybe.
My approach thus far:
Using SNMP on the Cisco HSRP gateway IP: Nagios sends 2 arguments, IP of the host (which should be the source of the multicast), i.e. 172.18.25.101 Second argument is IP of the stream ($mroute), i.e. 239.101.0.1
snmpwalk -v 2c -c changed 172.30.0.1 1.3.6.1.3.59.1.1.2.1.4 | grep $mroute | sed -e 's/.*IpAddress: //'
A few if's later, and I have, if the stream is on the network, if the multicast I sid for matches the host ip, or if not tells me where is is coming from. And exits correct for nagios.
Or so I thought. Generally it is working as expected, but randomly with some hosts the source IP is not expected and is something different, and when checking manually it is clearly not correct. I think maybe a topology change or something (we have quite a large network), and it's seen from the other gateway or... I'm not great on multicast sorry.
I'm pretty much stuck with the above part.
I then wanted to check that the video/audio was not frozen, I thought another check could be to use mplayer to dump the stream for 2 seconds to a file and do a check based on the size of the file. if it is very small then its probably frozen. But the stream will still send an image, so go with an audio check over a longer, say 10 second period. The more I thought about this, the more I thought "there must be a better way"...
IPTV is pretty big these days, how are people monitoring multicast streams.
Thanks very much.
Have you considered using the IP-MROUTE-STD-MIB rather than the IGMP MIB? You can get statistics on a per-mroute basis - which will give you a much better view of the source in particular. There's also a set of Cisco extensions to this MIB that can provide more in the way of platform-specific info. One item you can potentially look for is a substantial difference in counters on your various routers through the path of the mroute. Some delta is to be expected but this is would be a good place to track thresholds.
For tracking streams freezing there's a pretty easy answer: ip multicast heartbeat (http://www.cisco.com/en/US/docs/ios/12_2/ip/configuration/guide/1cfmulti.html#wp1003131). You can configure a given router to throw an SNMP trap if no packets are seen on a configured multicast group for 10 seconds.
There is also a feature called mrm (multicast route monitor) that can be called from the Cisco CLI to set up and track synthetic multicast groups. You'd likely want to use EEM or similar to call it periodically and then throw a trap or syslog if it doesn't behave normally. This is also a good troubleshooting tool.
Also - just as you (should) monitor for changes in IGP adjacency, so too should you track on PIM. Events like neighbor state changes, elections, etc can indicate instability in the tree. It's not -necessarily- a big deal in all cases but should generally be quiet on a stable network.
I'm not sure which supervisor you're running in your 4500's, but some of the more recent models support netflow for multicast. This would give you a much more granular and global view of multicast performance and would naturally lend itself to statistical trending, storage, etc.. definitely a good way to go.
I hope this helps-