A Guide to ArcFM Application Metrics

Version 3
Visibility: Open to anyone

    This document describes our ArcFM Application Metrics efforts with respect to the ArcFM 10.2.1 release and the 10.1.1 QIP patch (#581 and up). Application Metrics is not related to the Responder Telemetry Service.

     

    What is it?

     

    Application Metrics captures usage information. For example, how many times a particular autoupdater fires, or how long it takes a stored display to open for the first time in an ArcMap session.

     

    Similar to Microsoft’s Customer Experience Improvement Program, Application Metrics collects a low volume of usage data.

     

    Why should I participate?

     

    Help us improve customer experience.

     

    We get a better understanding of how you use our software.

     

    We may give you feedback that might improve your usage patterns (e.g., suggest a more efficient way to perform a frequently executed task).

     

    We may be able to identify and fix performance or usability problems even before they rise to the level of a tech support call.

     

    What’s collected?

     

    Data captured by Application Metrics is logged in different ways, depending on what it’s measuring.

     

    There are three types of log records. Each type includes certain basic information such as the machine name, the username, the domain, the time at which it was logged, and one or more timestamps.

     

    Per Occurrence

    An entry is logged each time the event occurs. There is one row per event, and each record shows the time at which the event occurred. This type of event is useful for gauging when an event occurs, such as before or after another particular event. Events like these can help you understand something about the timing or sequence of events, not simply the number of times it happens (how many times a button is pressed is a statistic).

     

    Duration Per Occurrence

    Each time the event occurs, its duration is logged. All of the details captured in the Per Occurrence record type are also captured in this record type. It provides the most detailed log entries of all three record types.

     

    An example of a duration-per-occurrence event is the time required for the Identify Tool to return its results.

     

    Summary Statistics

    One entry is logged when the application shuts down. This record type includes the total number of times a numerical value was recorded (e.g., number of features processed) while the application was running, plus the min, max, mean and variance of the recorded values. The very first recorded value is also reported. This is the least detailed entry. It’s designed to conserve logging bandwidth by issuing a summary at the end rather than a separate entry each time the instrumenting code is reached.

     

    An example of a summary statistic entry is the execution of an autoupdater. We currently keep a separate set of summary statistics for each distinct AU that executes.

     

    The following table details exactly what information we collect.

    What We Are MeasuringLogging Type
    Application - open (additional info logged: product version, build number)Per Occurrence
    Application - shutdown - associated with corresponding application start eventPer Occurrence
    Esent Database Recycle - number of times the Esent database is recycledSummary Stats
    Esent Database Delete - number of times the Esent database is deletedSummary Stats
    Esent Database Dirty Shutdown Exception - includes exception messagePer Occurrence
    Cathodic Protection Window - on createPer Occurrence
    Attribute Editor - on createPer Occurrence

    Attribute Editor Update Button - every time a user clicks the update button

    Per Occurrence
    Attribute Editor Edit Shape Button - every time a user clicks the update shape buttonPer Occurrence
    Attribute Editor Edit Blob Button - every time a user clicks the buttonPer Occurrence
    Identify Tool - the Method parameter indicates which caller invoked it (OnMouseDown, DoubleCircleGestured, TapGesture)

    Duration Per Occurrence

    Locator Tool Reset Button - every time a user clicks the reset buttonPer Occurrence
    Locator Tool Searching - name of selected item in combo boxDuration Per Occurrence
    Feeder Export GP Tool - number of input feedersSummary Stats
    Feeder Export GP Tool - number of seconds to fetch object IDs for one feederSummary Stats
    Feeder Export GP Tool - number of seconds to export all the features of one feederSummary Stats
    Feeder Workspace Extension - initialization (additional info logged:  workspace type, path)Per Occurrence
    Feeder Workspace Extension - initialization exception (additional info logged:  workspace type, path, exception message)Per Occurrence
    Feeder Workspace Extension - shutdown (additional info logged:  workspace type, path)Per Occurrence
    Feeder Cache - number of de-energized features added to the feeder info cacheSummary Stats
    Feeder Cache - number of milliseconds to compute the set of de-energized features to be added to the cacheSummary Stats
    Feeder Cache - number of energized features added to the feeder info cacheSummary Stats
    Feeder Cache - number of milliseconds to compute the set of energized features to be added to the cacheSummary Stats
    Autoupdater execution time in secondsSummary Stats
    Feeder Manager 1.0 initialization - number of seconds, # of sources, # of subsourcesPer Occurrence Duration
    Feeder Manager 2.0 initialization, sources - number of seconds to initialize sources only, # of sourcesPer Occurrence
    Feeder Manager 2.0 initialization, subsources - number of seconds to initialize subources only, # of subsourcesPer Occurrence
    Open stored display - time in secondsSummary Stats
    MMPropertiesExt - Startup time in secondsSummary Stats
    CSM Trace toolbar intialization timePer Occurrence Duration
    Electric Trace toolbar initialization timePer Occurrence Duration
    Gas Trace toolbar initialization timePer Occurrence Duration
    Gas Isolation Trace toolbar initialization timePer Occurrence Duration
    Water Trace toolbar initialization timePer Occurrence Duration
    Water Isolation Trace toolbar initialization timePer Occurrence Duration
    ArcFM toolbar initialization timePer Occurrence Duration

    What about security?

     

    All identifying information is anonymized (hashed). So while we could relate data points with unique domains and users, we do not know who they are. In this fashion we can identify whether a problem is unique to a small set of users (e.g., a single client) or whether it’s a larger problem.

     

    Is performance affected?

     

    During runtime

     

    The following table should give you an idea of the small effect that ArcFM Application Metrics has on ArcFM during runtime:

    image00.png

    The measurements shown above were made by inserting stopwatch code into the Application Metrics logging methods. The test consisted of the following:

     

    1. Start ArcMap
    2. Logging into ArcFM on the Minerville file geodatabase sample data (after configuring for Feeder Manager 1.0 only, not Feeder Manager 2.0).
    3. Open a stored display.
    4. Start an editing session.
    5. Use the Select By Feeder tool to select all features belonging to a single feeder.
    6. Use the Find Feeder tool to zoom to and select all features belonging to a different feeder.
    7. Open all phases of a circuit breaker (i.e., a circuit source).
    8. Quit ArcMap without saving edits and without saving the map document.

     

    Note that these figures report only the direct impact of Application Metrics logging activity upon the main thread of the ArcMap application. The Application Metrics logging functions run on background threads. None of the time spent in those background threads is included in the measurements recorded above. That’s because the impact of that time upon perceived performance is likely very low on any multi-core machine and is difficult to quantify.

     

    At application shutdown

     

    Under certain circumstances, users may experience a one- to three-second delay between the moment the user acts to shut down ArcMap and the moment at which the process actually exits.

     

    How do I configure it?

     

    The choice to participate in ArcFM Application Metrics at 10.2.1 is not made by the user during the installation process. ArcFM 10.2.1a and later provide the option to participate in ArcFM Application Metrics on installation.

    appmetricsinstaller.png

     

    After installation, you can enable or disable ArcFM Application Metrics using Desktop Administrator. ArcFM 10.2.1 users must enable ArcFM Application Metrics through Desktop Administrator as the default is to disable reporting. You can access Desktop Administrator via the Start menu or from ...\Program Files (x86)\Miner and Miner\ArcFM Solution\LicenseServer\DesktopAdmin.exe:

    image01.png

    This setting only enables Application Metrics for ArcFM Desktop. In the future, the team expects to develop the ability to selectively enable Application Metrics for different components and products, such as Geodatabase Manager and ArcFm Server.