A Discussion on Computer Resources when Moving to the ArcGIS Parcel Editing Solution (the Parcel Fabric)

Executive Summary

The Parcel Fabric technology behind ESRI’s Parcel Editing Solution, differs from the standard feature classes used within ESRI’s geodatabases in many ways. The Parcel Fabric, looking to develop a more structured and integrated land records solution, incorporates all the traditional thematic layers used by land records, together with auxiliary  functional feature classes to provide a more complete solution that provides for creation, maintenance, history, metadata and adjustment layers required in a production environment to ensure the “mappers” have at their immediate disposal a complete resource for cadastralists. This structure, while being optimized for land record maintenance places a much more rigorous demand on computer resources. This short discussion will explore some of the demand points.

Integrated Feature Classes

The Parcel Fabric, in order to integrate the various thematic layers and make the validation of “topology” between the traditional features, manages the features differently. Rather than maintaining a multitude of feature classes, each having separate. discrete thematic feature classes, groups all features in the Parcel Fabric by geometry type, or “the type of geometric elements they represent such as polygon areas, line features and point features” rather than “the type of geometric features that represent, such as PLSS Township, Sections, Subdivisions, Lots, Tax Parcels, Encumbrances and Other areas, along with each layers corresponding lines and points”.  By reducing the number of feature classes included in the Parcel Fabric, it simplifies the structure, ensuring all features share the same spatial reference and connectivity, but using attribution to differentiate between the elements they represent.

This unification of geometry types complicates the search and display operations of the Parcel Fabric by having a greater number of features that must be searched through for record retrieval.  This integration is one reason why the creation, optimization and constant maintenance of the spatial indices with the feature dataset containing the Parcel Fabric is critical.

Relational Data Structure

Within the Parcel Fabric, all geometric feature classes, together with additional auxiliary operation feature classes and tables, are related to one another.  The Parcel Fabric can be thought of as a “geometric network” for  polygons.  Each polygonal record contains a complete set of lines that were used to create the polygon boundary, together with a corresponding record in the plan table containing the source data metadata and relates to all corner points that contain information about how the polygon corners are “linked” or “joined” together to define the spatial location and geometry of the polygon. As such, operations requiring the retrieval of a single record requires that that single record, and all related features must be retrieved to provide the Cadastralist with necessary information of every element related to that polygon. This intensified retrieval of single, al all related records places a much greater demand on the ability of the Parcel Fabric to index, search and retrieve the database records.

Editing Integrated Records

Because all records in the Parcel Fabric are related to corresponding line and corner records, editing the Parcel Fabric in a multi-user editing environment such as ESRI’s SDE versioning environment is highly transactional.  Within the SDE editing environment all changes to the tables, including adding records, deleting records or modifying records place “temporary” records in the “Change or Delta” tables that represent the “pre-committed”, or “posted”, data changes.  Since all records have these multitude of of related data associated with them, the Parcel Fabric will create or delete these associated features every time any change occurs to a polygonal record. For example, a simple modification of the location of a single point requires that all polygons, including polygons that are considered on other “layers” such as PLSS layers, subdivision layers, lot layers, easement layers, along with those lines connected to that corner, must be deleted from the Change table and added back into the change table to reflect that one simple move.  While that one simple move in a simple feature class may be represented by a delete and an add, in the Parcel Fabric, this one change may be represented by 30 or so deletes and 30 or so adds into these same Change tables. The more the data is edited in the Parcel Fabric, the larger and more cumbersome the Change tables become.  

For this reason, ESRI restricts the levels of versioning in the Parcel Fabric to only one level and highly recommends that maintenance on the Change tables be performed daily using the “analyze” geoprocessing tool to reduce and compress the number of redundant “state’ records contained in the Change table.  ESRI also recommends that all versions be reconciled and posted to the Default database state as quickly as possible.

Possible Implementation Scenarios

Taking into consideration the impact of the Parcel Fabric on the operations of the enterprise Geodatabase, there are several implementation recommendations.  

Segregated Implementation - The purpose of the Parcel Fabric is solely the creation, maintenance and production of land record information and the publication of this data should NEVER be performed from this highly integrated data structure (the Parcel Fabric). Rather, the data should be extracted, transformed and loaded into a publication data structure, while this implementation is often performed on the same server, just into separate feature datasets, one alternate suggestion some of our clients have taken is to segregate the entire Parcel Fabric structure and the geodatabase containing the Parcel Fabric onto a separate database server. The exact configuration and implementation strategy is completely depend upon the organization's infrastructure and deployment strategy.

Optimized Hardware - Since the Parcel Fabric technology is highly transactional in nature, optimization of Input / Output is critical to success. This points to the recognition that much of the demand of resources for the Parcel Fabric in a multi-user database is not computational, but is the increased demand on the searching and retrieval of all the related records contained with the Parcel Fabric.  We have had several clients successfully realize the greatest return on investment by configuring the data server sufficient memory and with solid state drives to optimize the I/O functions contained therein.

Summary

Governmental agencies using taxpayer funds to pay for their internal systems need to justify expenditures, but, in the case of the Parcel Fabric, it often makes more sense to spend additional funds to optimize the IT configuration for the use of the Cadastralists.

 

The File Geodatabase and the Parcel Fabric

We have seen an increase in the number of small and medium sized organizations interested in maintaining their parcel data in the esri ArcGIS Parcel Editing Solution (the Parcel Fabric.) Many of these organizations have only one person editing the data and using the file geodatabase structure is a great solution for them.

When using a larger, multi-user, enterprise geodatabase such as SDE, it is important to maintain the spatial indexes in an optimal state so that overtime a user searches or changes the display, the system responds as quickly as possible.  Few people realize that these indexes are just as important for the file geodatabase. While ArcGIS for Desktop does a fairly good job of maintaining these indexes, when using the Parcel Fabric technology, it is often a good idea to rebuild these spatial indexes of the feature classes inside the Parcel Fabric manually to ensue they are as optimized as possible.

This first screenshot shows where the feature classes that are part of the Parcel Fabric are stored.

This next image shows the steps to rebuilding the spatial index, remember, this must be performed for each of the five (5) feature classes in the Parcel Fabric, (ParcelFabric_Control, ParcelFabric_LinePoints, ParcelFabric_Lines, ParcelFabric_Parcels, and ParcelFabric_Points.)

The steps are:

1.) In ArcCatalog, expand the ParcelFabric.

2) Right Click on each of the feature classes in the Parcel Fabric. Remember,  the Parcel Fabric_Plans is a table and not a feature class so there is no spatial index.

3) Under the Indexes Tab on the Feature Class Properties dialog box, click on the Delete button to remove the existing index.

4) Immediately after the existing index is deleted, click on the Create button to recreate a new index.

Click OK and enjoy the new found speed in the Parcel Fabric.

Happy Holidays 2015

We at Panda Consulting wish very everyone the brightest of Holidays, a Merry Christmas and a sincere hope that we all have a Joyous, Healthful and Happy New Year.

We are thankful for the many new friends we met this year along the way and the many old friends that allowed us to continue helping throughout the year.

We continue to maintain our focus on Land Records and the esri ArcGIS Parcel Editing Solution (the Parcel Fabric), both training our friends in how to effectively use this new, more effective approach to parcels and guiding them in their transition into this new technology.

Enjoy and have a Merry Christmas and a Happy New Year.

The Keys to a Successful Transition to the Parcel Editing Solution (the Parcel Fabric)

Having performed several dozen migrations for large and small clients across the United States with everything from hundreds of CAD files to topologically valid feature classes already in SDE and provided training to hundreds of students across the United States about the Parcel Fabric technology in the ESRI ArcGIS Parcel Editing Solution, I have often been asked what differentiates those organizations that successfully transition into the ArcGIS Parcel Editing Solution from those that struggle during the transition.  While it is hard to pinpoint any single characteristic of the successful transitions, but I can say there are certain traits that help. 

What Should my Source Data Look Like?

First, success with the transition is less technology-driven and more personnel-driven.  While having feature classes with good valid topology does help - the technical challenges of converting data from any source is fairly easily overcome. There are ways to ensure the source data that will be used to construct the various polygon types will migrate correctly, but these tools all work against the features while they are still stored as topology feature classes.  There are several add-ins to help and there is a multitude of geoprocessing tools to "fix" the source data prior to the migration and any good ESRI business partner that is knowledgable about the migration clearly knows what to look out for and what should be done with the data to ensure the migration is successful. However, it is not the data that makes the transition successful or not, it is the personnel responsible for the everyday maintenance that will determine how quickly the successful transition occurs

Should I Do the Data Migration Myself?

Several times Panda Consulting has been contacted by organizations that are either struggling to migrate their data, or, have migrated their data and struggling as a result of that migration. The migration of your source data into the Parcel Fabric is not trivial and it is only after several learning attempts that one can fully understand how to make a successful migration. There is everything from correcting incorrect topology - to "fixing" curves that have been "exploded" through earlier migration in and out of shapefiles - to creating valid road polygons to ensure the parcels do not experience coordinate creep - to fully ensuring all attribution is correctly migrated. It is important to have the help of someone who has made this migration before.  I liken this to working on your car, while it is possible for you to perform many maintenance tasks with your vehicle, when it comes time to perform major maintenance tasks, it is time better spent by having an expert perform these tasks for you, especially since you should only need to perform this migration once.  

The Impact of Personnel

It is the flexibility / adaptability of the personnel, the "mappers",  to transition the way in which they think about what they do and the products they create that will determine how well the transition occurs. If the people that will be working with the data every day are open to the new tools and the new ways to look at the data, they can quickly make the transition. I have seen this transition from initial exposure to the Parcel Editing Solution to being "comfortable" with the approaches, tools and confident in the ability to effectively manage the data range from 30 days to 9 months, with 6 months being the average amount of time it takes for the cadastralists to be completely comfortable with the transition.  

An early indication of the flexibility is how comfortable the mappers are in having their data appear differently in the ArcMap map document. If they can quickly adapt to viewing the data using the default table of contents when loading the Parcel Fabric, even with minor "tweaks", the faster they can recognize that the quality of the data is more important that then way it looks.  Within the Parcel Editing Solution, they data is presented one way for efficiency in editing and another way for publication of data and the ability to recognize this new way of looking at the data is critical for the successful transition.

The Tools

The Parcel Fabric technology used in the Parcel Editing Solution uses "composite" features to represent the parcels.  Since each composite feature consists of a single polygon, all lines associated with that polygon, ties to corner points that are used to join parcel corners together and links to control points (in addition to source document information, i.e. "plans"), the standard editing tools used to maintain simple feature classes do not work against these composite features.  Instead, there are a new set of Parcel Fabric editing tools that manipulate the composite features. These new Parcel Fabric editing tools, because they work against multiple features at the same time are actually more efficient that the standard editing tools.  How adaptable the mappers are in making the transition also differentiates the successful transition from this that struggle.  It is ironic that sometimes those clients that have never used ArcGIS standard editing tools, nor done any parcel mapping in ArcMap can actually make the transition faster than those clients that have been using COGO and advanced parcel mapping techniques for years.

Continued Support

Due to the extended time and continued exposure to the tools before being completely comfortable, it is important that new users have access to continued support.  This support can be either the Land Records Meetup provided by ESRI that meets on a regular basis, the Virtual Panda services offered by Panda Consulting or the bi-weekly Panda Parcel Fabric Forum, an online meeting for people who wish to learn more about the Parcel Editing Solution and share best practices for using the tools. The Panda Parcel Fabric Forum is available by invitation only so, please contact us if you wish to be invited.

From Cadastralists to Curators

The long term impact of transitioning into the Parcel Editing Solution is a change in the focus of the cadastralist from performing cartography, i.e. creating "maps", into performing digital curation of land records data. There is less focus on how the map looks and a greater focus on making the data correct. Many little things that could be "fudged" to make the map look valid become exposed during the migration of the data and become visible inside the Parcel Editing Solution. Those mappers that can make that transition have proven to be the most successful in successfully adopting and taking advantage of the benefits of the ArcGIS Parcel Editing Solution.

Measurements versus Monuments

One of the more interesting effects of transitioning the way in which you manage your land records into the ESRI ArcGIS Parcel Editing Solution (the Parcel Fabric) is the transition from mapping parcels according to the measurements listed in the legal descriptions to mapping parcels according to the monuments called out in those legal descriptions.

Traditionally, and I am not sure whether this is the result of the limited data structures or just a misunderstanding into legal description interpretation, but, most mappers input the bearings and distances cited in the legal descriptions with little attention being paid to the accompanying monument calls.  You know.... "thence along the North right of way line" or "to the southeast corner of Lot 47" or "47.08 feet from the Northwest Section corner."

These calls, while they appear to be minor in nature, are actually more important that the actual measurements in the legal description.  For example, if a course is described as "thence North 26 degrees 15 minutes 26 seconds East, for a distance of 100 feet to the southeast corner of Lot 47", even if the actual bearing and distance to the southeast corner of Lot 47 is North 30 degrees East, a distance of 105 feet to the monument found located on the southeast corner of Lot 47, the property corner is the southeast corner of Lot 47 and the location of the monument supersedes the measurement called in the description and should be used for mapping.

These calls are commonly referred to as "monuments" and can exist in many forms: They can be natural monuments such as trees, streams, and lakes) or artificial monuments such as fences, walls, roads, streets or railroads. They can also be monuments described as the boundaries of adjacent parcels of land. The exact location of these monuments is considered "superior" to and carries a great deal of strength in determining the location of any specific corner, especially if that monument is called out in the legal description. 

Just a short note, for the monument call to be valid, it must be included in the legal description itself and cannot be inferred.

Monuments in the ArcGIS Parcel Editing Solution (the Parcel Fabric)

The Parcel Fabric is designed to enforce the manner in which the corner locations are defined, allowing the mapper to input the bearings and distances as cited in the description, but also locating the parcel corners on the existing mapped corners or "monuments".  This is accomplished using the "join" function inside the Parcel Fabric.  This "join" function allows the Parcel Fabric to locate a parcel corner so it is contiguous to and "along the right of way" or located at the "South East corner of Lot 47" or at "Section Corner" while maintaining the measurement information found in the source legal description.

This one function changes the way in which we handle the various inconsistencies contained in the legal descriptions, whether these inconsistencies are caused by differences in the precision of the measuring standards, differences in opinions of the locations of other monuments or improperly written deeds.  The "joining" functions allows us to analyze and map the "intent" of the description without being fixated on the measurements contained within. In my opinion, the one change is powerful and should be sufficient enough to convince us to begin managing our land records and parcel information in this new way.

I would be interested in hearing your views below.

Happy Holidays

We at Panda Consulting wish you and yours the brightest of Holidays and a sincere hope that we all have a Joyous, Healthful and Happy New Year.

It has been a while since I have posted anything to our blog - we have been very busy this year and I was not able to share as much as I had hoped.  To review some of the projects on which we have been able to help our Clients, please head over to our Projects page for a quick view.

Enjoy and have a Merry Christmas and a Happy New Year.