This document describes our ArcFM Application Metrics efforts with respect to the ArcFM 10.2.1 release and the 10.1.1 QIP patch (#581 and up). Application Metrics is not related to the Responder Telemetry Service.
What is it?
Application Metrics captures usage information. For example, how many times a particular autoupdater fires, or how long it takes a stored display to open for the first time in an ArcMap session.
Similar to Microsoft’s Customer Experience Improvement Program, Application Metrics collects a low volume of usage data.
Why should I participate?
Help us improve customer experience.
We get a better understanding of how you use our software.
We may give you feedback that might improve your usage patterns (e.g., suggest a more efficient way to perform a frequently executed task).
We may be able to identify and fix performance or usability problems even before they rise to the level of a tech support call.
Data captured by Application Metrics is logged in different ways, depending on what it’s measuring.
There are three types of log records. Each type includes certain basic information such as the machine name, the username, the domain, the time at which it was logged, and one or more timestamps.
An entry is logged each time the event occurs. There is one row per event, and each record shows the time at which the event occurred. This type of event is useful for gauging when an event occurs, such as before or after another particular event. Events like these can help you understand something about the timing or sequence of events, not simply the number of times it happens (how many times a button is pressed is a statistic).
Duration Per Occurrence
Each time the event occurs, its duration is logged. All of the details captured in the Per Occurrence record type are also captured in this record type. It provides the most detailed log entries of all three record types.
An example of a duration-per-occurrence event is the time required for the Identify Tool to return its results.
One entry is logged when the application shuts down. This record type includes the total number of times a numerical value was recorded (e.g., number of features processed) while the application was running, plus the min, max, mean and variance of the recorded values. The very first recorded value is also reported. This is the least detailed entry. It’s designed to conserve logging bandwidth by issuing a summary at the end rather than a separate entry each time the instrumenting code is reached.
An example of a summary statistic entry is the execution of an autoupdater. We currently keep a separate set of summary statistics for each distinct AU that executes.
The following table details exactly what information we collect.
What We Are Measuring Logging Type Application - open (additional info logged: product version, build number) Per Occurrence Application - shutdown - associated with corresponding application start event Per Occurrence Esent Database Recycle - number of times the Esent database is recycled Summary Stats Esent Database Delete - number of times the Esent database is deleted Summary Stats Esent Database Dirty Shutdown Exception - includes exception message Per Occurrence Cathodic Protection Window - on create Per Occurrence Attribute Editor - on create Per Occurrence
Attribute Editor Update Button - every time a user clicks the update button
Per Occurrence Attribute Editor Edit Shape Button - every time a user clicks the update shape button Per Occurrence Attribute Editor Edit Blob Button - every time a user clicks the button Per Occurrence Identify Tool - the Method parameter indicates which caller invoked it (OnMouseDown, DoubleCircleGestured, TapGesture)
Duration Per Occurrence
Locator Tool Reset Button - every time a user clicks the reset button Per Occurrence Locator Tool Searching - name of selected item in combo box Duration Per Occurrence Feeder Export GP Tool - number of input feeders Summary Stats Feeder Export GP Tool - number of seconds to fetch object IDs for one feeder Summary Stats Feeder Export GP Tool - number of seconds to export all the features of one feeder Summary Stats Feeder Workspace Extension - initialization (additional info logged: workspace type, path) Per Occurrence Feeder Workspace Extension - initialization exception (additional info logged: workspace type, path, exception message) Per Occurrence Feeder Workspace Extension - shutdown (additional info logged: workspace type, path) Per Occurrence Feeder Cache - number of de-energized features added to the feeder info cache Summary Stats Feeder Cache - number of milliseconds to compute the set of de-energized features to be added to the cache Summary Stats Feeder Cache - number of energized features added to the feeder info cache Summary Stats Feeder Cache - number of milliseconds to compute the set of energized features to be added to the cache Summary Stats Autoupdater execution time in seconds Summary Stats Feeder Manager 1.0 initialization - number of seconds, # of sources, # of subsources Per Occurrence Duration Feeder Manager 2.0 initialization, sources - number of seconds to initialize sources only, # of sources Per Occurrence Feeder Manager 2.0 initialization, subsources - number of seconds to initialize subources only, # of subsources Per Occurrence Open stored display - time in seconds Summary Stats MMPropertiesExt - Startup time in seconds Summary Stats CSM Trace toolbar intialization time Per Occurrence Duration Electric Trace toolbar initialization time Per Occurrence Duration Gas Trace toolbar initialization time Per Occurrence Duration Gas Isolation Trace toolbar initialization time Per Occurrence Duration Water Trace toolbar initialization time Per Occurrence Duration Water Isolation Trace toolbar initialization time Per Occurrence Duration ArcFM toolbar initialization time Per Occurrence Duration
What about security?
All identifying information is anonymized (hashed). So while we could relate data points with unique domains and users, we do not know who they are. In this fashion we can identify whether a problem is unique to a small set of users (e.g., a single client) or whether it’s a larger problem.
Is performance affected?
The following table should give you an idea of the small effect that ArcFM Application Metrics has on ArcFM during runtime:
The measurements shown above were made by inserting stopwatch code into the Application Metrics logging methods. The test consisted of the following:
- Start ArcMap
- Logging into ArcFM on the Minerville file geodatabase sample data (after configuring for Feeder Manager 1.0 only, not Feeder Manager 2.0).
- Open a stored display.
- Start an editing session.
- Use the Select By Feeder tool to select all features belonging to a single feeder.
- Use the Find Feeder tool to zoom to and select all features belonging to a different feeder.
- Open all phases of a circuit breaker (i.e., a circuit source).
- Quit ArcMap without saving edits and without saving the map document.
Note that these figures report only the direct impact of Application Metrics logging activity upon the main thread of the ArcMap application. The Application Metrics logging functions run on background threads. None of the time spent in those background threads is included in the measurements recorded above. That’s because the impact of that time upon perceived performance is likely very low on any multi-core machine and is difficult to quantify.
At application shutdown
Under certain circumstances, users may experience a one- to three-second delay between the moment the user acts to shut down ArcMap and the moment at which the process actually exits.
How do I configure it?
The choice to participate in ArcFM Application Metrics at 10.2.1 is not made by the user during the installation process. ArcFM 10.2.1a and later provide the option to participate in ArcFM Application Metrics on installation.
After installation, you can enable or disable ArcFM Application Metrics using Desktop Administrator. ArcFM 10.2.1 users must enable ArcFM Application Metrics through Desktop Administrator as the default is to disable reporting. You can access Desktop Administrator via the Start menu or from ...\Program Files (x86)\Miner and Miner\ArcFM Solution\LicenseServer\DesktopAdmin.exe:
This setting only enables Application Metrics for ArcFM Desktop. In the future, the team expects to develop the ability to selectively enable Application Metrics for different components and products, such as Geodatabase Manager and ArcFm Server.