8 Replies Latest reply on Mar 12, 2014 9:40 AM by Francesco Giovinazzo

    Automated Reconcile and Post Processes

    Ed Blair

      I heard at LINK (@link 2014) last week that some companies are using scheduled python script routines to accomplish the same thing that one would accomplish with the ArcFM GDBM (Geodatabase Manager).   That is, schedule reconcile/post/compress processes so that individual users don't have to.

       

      Can anyone comment on the pros and cons of this approach?

       

      Thx
      Ed

        • Re: Automated Reconcile and Post Processes
          Jim Kivela

          That's not exactly what we do. Instead we do weekly batch reconciles (we don't automatically post anything, not sure why anyone would want to do that), using a C# Windows forms app that is a rewrite I did of ESRI's old developer sample Batch Reconcile (from VB6 days).  It does contain functionality to post/delete versions if you choose, although we don't use those.  Since it's a Windows forms app you do have to manually launch the process.  I guess I could've written it as a console app instead and then scheduled it with Windows, but logging in once a week and launching it is not a big deal so it's no problem.  Process has worked fine for us, we're small enough that one batch reconcile a week is sufficient.

            • Re: Automated Reconcile and Post Processes
              brownkj

              OK, Jim, I'll play devil's advocate.

              We have a "posting officer" who posts completed Designer designs, but also does other clean up on the Electric side using Session Manager.  We also have a Gas editor who does all the as-built editing (not in Designer yet).  I don't have time to pre-check every session that gets posted, so instead I have an auto-post process for sessions only.  Then on the backend I have a number of python cleaning and QA scripts to catch anything that slips through.

              My scheduled (every 15 minutes) python script for auto-post uses SQL to find sessions ready to post, then runs arcpy.ReconcileVersion_management to post it.  Then there's version clean up using arcpy.DeleteVersion_management and some SQL to clean up session rows also.

              I also have a nightly scheduled batch reconcile that goes through all "reconcilable" versions (excludes replica versions) with arcpy.ReconcileVersion_management (NO_POST) and then arcpy.Compress_management at the end.

              I prefer the visibility python gives me for these processes.  No custom ArcObjects code, but you do have to be very careful with the session clean up parts.

                • Re: Automated Reconcile and Post Processes
                  Jim Kivela

                  If that works fine for you guys, then it's a good process.  I personally just wouldn't want anything to ever be posted automatically without someone looking at it first. Some items I'm sure can be checked programatically but other things are visual or need some human logic applied, at least for how we operate.  We have a long list of items that staff who post are supposed to go through each time before they post a design up.  But I do realize that we're fairly small so they volume of work coming through my group is not that great, so we can have someone take the time to look at everything that gets posted (and yet some things still slip through anyway).  Larger organizations don't have that luxury, I'm sure.

                  • Re: Automated Reconcile and Post Processes
                    Ed Blair

                    Kevin -

                     

                    Just curious.  Is your python-based process based on a sample that Esri distributes?  I think they've got one on their developer samples site.

                     

                    Ed

                    • Re: Automated Reconcile and Post Processes

                      It sounds like your scripts could benefit from any Session Manager specific geoprocessing tools we might be able to add to ArcFM. In particular, the Orphaned Version Cleanup tool could be a good GP tool to have as you could just mark the newly posted sessions as 'orphaned' in your batch process and let our code take care of cleaning up things like session data, and session stored displays, saved Targets tab data, etc. Thanks for sharing some insight into your process!

                  • Re: Automated Reconcile and Post Processes
                    Francesco Giovinazzo

                    We are using two different solutions:

                     

                    In the first environment, where users have an sde.Default version for the data, and they create a personal version, edit it, then rec&post, we provided two buttons to automate the createversion-rec&post actions

                     

                    In the second environment, we have cascade versions, one for each day in the year, starting from default (wich actually is 01/01/2013, and the first child version is 01/02/2013, and so on). For this environment, we created a Windows Service wich scans the version lineage for versions that needs to be reconciled (i.e. wich had been edited after the last rec); when a version needs to be reconciled, the service starts a batch process wich handles the reconcile and geometric checks to ensure network connectivity. Starting a batch process allows for full memory cleanup after the process, the reconcile is very memory intensive and subject to memory leaks (so you cannot reconcile too many times in the same ArcMap version). There is also a scheduled batch wich compress every sunday.

                     

                    Everything was coded in .NET C#