3 Replies Latest reply on Mar 26, 2018 5:25 PM by Shawn Leingang

    Responder 10.2.1d with RxIf causing network flood during larger outages

    Aimin Fan

      Hi All,

      Is there anybody experienced network flooding when any bigger outage happen?

      I have responder 10.2.1d with RxIF setup and AMI works fine with responder except when there is a larger outage(more than 3-400 customers), it causes Our entire local network interrupted, sometime triggers SQL server fail-over.

      This created serious issue since all our systems including CIS tied up with our network.

       

      I appreciate any suggestion and hints.

      Thanks

       

      Aimin

        • Re: Responder 10.2.1d with RxIf causing network flood during larger outages
          Dave Gallelli

          Hello Aimin.  Sorry to read of your problems.  I'm trying to understand the scenario.  Not sure if you really meant to write 3-400 customers.  That would not constitute a large outage.  Did you mean to write 4000?  For various utilities we have tested much larger outages - even up to 100,000 in one hour - without issue.  Can you duplicate the problem in your test environment (if you have one)?  Or is it only occurring in production?  Have you actually detected that the network becomes saturated in this scenario?

           

          Thanks,

          Dave

            • Re: Responder 10.2.1d with RxIf causing network flood during larger outages
              Aimin Fan

              Thank you Dave for taking my question.

              It is the production not dev, I don’t have dev RXIF set up yet.

              And 3-400 Calls sometimes does trigger the issue, but the more customer/Call the longer interruption will presents.

              I can recreate by sending a few hundreds of outages by jmeter.

              I am not sure if the network become saturated or not , but our entire network get interrupted, include our phone system.

              We are trying to get some help from Network gear vendor but I notice once responder server/services shut down, the interruption stops.

               

              Thanks

              Aimin

                • Re: Responder 10.2.1d with RxIf causing network flood during larger outages
                  Shawn Leingang

                  It might be a good idea to have the networking hardware monitored and traffic monitored from the switches to determine what type of traffic it's struggling with; and see if there is a configuration change the vendor can make to handle it better.  Responder does generate a lot of traffic with it's design, but it's never been a problem for any type hardware vendor already in place with clients for past implementations.  Some clients have created VLAN's or other methods of isolation to limit the impact of other users and maybe it's something that can be implemented here as well.  Enabling of IGMP snooping can also limit the amount of multicast traffic being forwarded and should be enabled by the vendor too.