Accessing More CAD and GIS Data within AutoCAD® …aucache.autodesk.com/au2011/sessions/5176/class_handouts/v6_v3_GS...Accessing More CAD and GIS Data within AutoCAD® Map 3D 2 of

Accessing More CAD and GIS Data within AutoCAD® Map 3D

Dale Lutz– Safe Software Inc.

GS5176

With the FDO Data Access technology—the Autodesk geospatial data access platform—users are able to extend the format reach of their AutoCAD Map 3D installation. Attendees will see how the free FME FDO Provider saves time and provides users with the most up-to-date data by directly connecting to external data sources and bypassing the need to import and create redundant copies of their datasets. Using case studies and demonstrations, attendees will learn how to use the free FME FDO Provider to gain direct read access to nine popular formats, including ESRI, MapInfo, and GML/XML formats—all directly within AutoCAD Map 3D. By the end of this class, attendees will be able to quickly and easily improve access to spatial data for their organization.

Learning Objectives At the end of this class, you will be able to:

• See how the free FME FDO Provider allows you to directly connect to external data sources in Map 3D

• Avoid using Import and making copies of external data in your DWG files

• Impress your colleagues with your knowledge of the variety of data formats that the free FME FDO supports

• Know some options for efficiently dealing with format and data restructuring workflows

About the Speaker Dale Lutz is the co-founder and Vice President of Software Development for Safe Software Inc., based in Surrey, British Columbia, Canada. Along with business partner Don Murray, Dale created FME®, the world leading spatial data transformation technology.

Dale is a Computer Science Honors graduate and a Masters recipient with specialty in Usability for Software Technology from the University of Alberta. Starting his career at MacDonald Dettwiler, Dale went on to co-found Safe Software in 1993.

As Vice-President of Safe Software, Dale uses his more than 25 years experience in software to guide the core development of FME, refining and optimizing the user experience in the GUIs


2

of the FME platform, and leading the integration of FME technology into third-party GIS applications.

Known for his expertise in data interoperability, Dale regularly speaks at conferences and tradeshows around the world, sharing his passion for making data accessible to those who need it.

You can email Dale at [email protected], follow him on Twitter at @DaleAtSafe, or read his thoughts on “All Things Data” at his blog http://blog.safe.com

FDO Technology and the Free FME FDO FDO is a data access technology developed by Autodesk and released to the Open Source community via OSGeo. It was first released in 2004 and has been part of several Autodesk products ever since. As written on http://fdo.osgeo.org/overview.html, the Feature Data Objects (FDO) API provides a generic, command-based interface to a number of back-end data source technologies for storing, retrieving, updating, and analyzing GIS (Geographic Information System) data.

There are two cooperating parts to any FDO-powered GIS system. The host application itself must implement, in an extendible way, the ability to dynamically discover and employ FDO data providers to supply data to it. Map 3D, Autodesk Infrastructure Map Server (formerly known as MapGuide Enterprise), and Autodesk Infrastructure Modeler are all examples of FDO client applications.

The second part of an FDO system is the data provider. There are many open source data providers, providers developed by Autodesk and distributed with their products, and providers made by third parties to further extend those products.

The Free FME FDO is an implementation of the FDO API powered by FME technology. It is important to note that while the FDO specification and many FDO providers provide for the editing of data in addition to data reading, the FME FDO implementation is read-only.

FME Technology The FME software platform facilitates the conversion, transformation, and integration of both spatial and non-spatial data. The full product includes a large library of formats (see following graph), a large library of data transformation options (such as attribute renaming, testing, filtering, reprojection, and other geometric operations), a graphical environment for creating data transformation workflows, and even a server option to centralize and automate data tasks.


3

The free FME FDO makes use of several of the FME formats and makes them available within the AutoCAD Map 3D environment as read only data sources. The commercial FME product can also be used within the FDO framework of Autocad Map 3D to provide read-only access to all the GIS, CAD, Spatial Database and raster formats supported by the full FME product.

In addition, through a mechanism known as custom formats, new formats can effectively be defined by combining existing FME formats with transformation operations authored in the FME Workbench product. The end result is that users of AutoCAD Map 3D can access data and see it in a manner directly usable by them, which may be significantly different than the original structure of the data that is in the original formats, all without making any copies of the data or having to do any manual restructuring themselves.

Data Access Options for AutoCAD Map 3D When faced with the need to access external data within the Map 3D environment, users have a number of choices. The traditional option is to use the Import menu item and bring a copy of the data into the DWG file, complete with Object Data attributes. (Aside: the technology supply the data format access is FME technology.) The major downside of this option is that a copy of the data has been made, so if the original changes, the burden of keeping up to date is placed upon the user, who must do a re-import. Furthermore, the import process is a manual one and can become tedious and time consuming if multiple files must be processed.

Another option is to use the full FME product to do standalone “Spatial ETL” (Extract, Transform, and Load) to bring data from the external sources into Map 3D DWG files, complete with Object Data attributes. This has the advantage of being able to be run in a batch mode, external from Map 3D, and significant data restructuring can be done while data is being


4

imported to increase end user productivity by supplying precisely the right view of the right data to the end user. The downside is that again, a copy of the data has been made. (Note: this same technology can be used to share Map 3D Object Data out to users in the format of their need.)

For some formats, the open source FDO providers mentioned earlier can provide the direct read (as well as edit) necessary for interacting with external data. However, not all formats are available, and within some formats, not every nuance of the format may be supported.

The commercial FME FDO Provider, which is part of the FME Desktop product suite, fills this potential gap by providing read-only access to a large number of CAD, GIS, Spatial Database, and raster formats, all built atop the respected FME format access library platform.

Lastly, the free FME FDO Provider supplies, without any additional cost, FDO read access to several very important and widely used formats. Because it is free, there is really no reason not to use it when the situation calls for it.

Free FME FDO Format Summary The free FME FDO supports seven formats for read-only access. Each of these formats is described in some detail below. These descriptions are taken from the FME Format reader/writer manual, which is a great source of information on a wide variety of CAD, GIS, Database, Raster, Point Cloud (LiDAR) and 3D formats.

Esri ArcInfo Coverage & Esri ArcInfo Export (E00) Reader

Overview A single E00 file describes a complete ArcInfo coverage. The file itself is actually an archive of several smaller files, referred to here as subfiles. Some of these subfiles have fixed names which do not vary from coverage to coverage, and follow a predefined data format. These are referred to as the standard subfiles.

The remainder of the subfiles contained within an E00 are the info files. These files may contain user-defined attributes, and have names which vary from coverage to coverage. The ways in which the names vary are discussed in Info Files.

E00 Quick Facts

Format Type Identifier E00 or ARCINFO Dependencies None

Dataset Type • Directory for ArcInfo Coverage reader • Directory for ArcInfo Table reader • File for E00 reader


5

Feature Type • Subdirectory base name for ArcInfo • File base name for E00

Typical File Extensions .e00 (.e01, .e02, ...) User-Defined Attributes Yes

Coordinate System Support Yes

Geometry Support Geometry Supported? Geometry Supported? aggregate no point yes circles no polygon yes circular arc no raster no donut polygon yes solid no

elliptical arc no surface no

ellipses no text yes line yes z values n/a none yes

Geography Markup Language (GML & GMLSF)

GML Overview GML is an OpenGIS® Implementation Specification. The GML specification defines an XML encoding for the transport and storage of geographic information. This specification can be found at the Open GIS Consortium website www.opengeospatial.org.

GML documents must be instances of a conforming application schema. Conforming application schemas are to be defined with the W3C’s XML Schema language.

GML Quick Facts

Format Type Identifier GML Dataset Type File

Feature Type Varies depending on the GML application schema

Typical File Extensions .gml, .xml User-Defined Attributes Yes Coordinate System Support Yes

Geometry Support


6

Geometry Supported? Geometry Supported? aggregate yes point yes circles no polygon yes circular arc yes raster no donut polygon yes solid no


ellipses no text no line yes z values yes none yes

MapInfo MIF/MID

MapInfo MIF/MID Overview MapInfo is a two-dimensional (2D) system with no provision for transferring elevation data for each vertex in a MapInfo feature. However, point features can define an elevation attribute to store their elevation.

MIF files store both feature geometry and attribution. A logical MIF file consists of two physical files, having the following file name extensions:

File Name Extension Contents

.mif Vector geometric data

.mid Attributes for the geometric data

These extensions are added to the basename of the MIF file.

The MapInfo reader and writer support the storage of point, line, polyline, arc, ellipse, rectangle, rounded rectangle, region (polygon), and text geometric data in .mif files. The MIF format also stores features with no geometry. Features having no geometry are referred to as having a geometry of none.

Each geometric entity present in a .mif file has display properties such as pen and brush width, pattern, and color. In addition, each entity has a row of attributes stored in an associated.mid file. A single .mif file contains many different types of geometry however, the associated attribute in the .mid file must have the same number and type of fields for each entity in the.mif file. The order of the entries in the two files is synchronized. For example, the second geometric entity in the .mif file has the attributes held in the second row of the .mid file.

The number and type of attributes associated with each entity is specified by the user. There must be at least one attribute field in the .mid file.


7

The following example shows a MIF file containing three region entities in it. Note that the second polygon contains a hole, and the third polygon is an aggregate of two disjoint polygons, one of which contains a hole. Each geometric entity in turn corresponds with one record in the attribute table.

MapInfo MIF/MID Quick Facts

Format Type Identifier MIF Dataset Type Directory or File Feature Type File base name Typical File Extensions .mif (.mid) User-Defined Attributes Yes Coordinate System Support Yes

Geometry Support


8

Geometry Supported? Geometry Supported? aggregate yes point yes circles yes polygon yes circular arc yes raster no donut polygon yes solid no

elliptical arc yes surface no

ellipses yes text yes line yes z values no none yes

MapInfo TAB

MapInfo TAB Overview MapInfo is a two-dimensional system with no provision for transferring elevation data for each vertex in a MapInfo feature. However, point features can define an elevation attribute to store their elevation.

MapInfo files store both feature geometry and attributes. A logical MapInfo file consists of several physical files, having the following file name extensions:

File Name Extension Contents

.tab The main file for a MapInfo table, it is associated with the appropriate DAT, MAP, ID, and IND files.

.dat, .dbf, .mdb,

.accdb, .xls

Tabular data for a table in MapInfo’s native format(.dat), dBASE format (.dbf), MS Access format (.mdb or .accdb) or MS Excel format (.xls). MS Access and MS Excel formats are only supported when using the MITAB reader.

.id An index to a MapInfo graphical objects (MAP) file.

.map Contains geographic information describing map objects.

.ind An index to a MapInfo tabular (DAT) file.

These extensions are added to the basename of the specified MapInfo file. Throughout the remainder of this chapter, references to “file” are references to the logical MapInfo file, not the multiple physical files that make it up.

The MapInfo reader and writer support the storage of point, line, polyline, arc, ellipse, rectangle, rounded rectangle, region (polygon), and text geometric data. The MapInfo format also stores features with no geometry. Features having no geometry are referred to as having a geometry of none.


9

Each geometric entity present in MapInfo has display properties, such as pen and brush width, pattern, and color. In addition, each entity has a row of attributes associated with it. A single MapInfo map file can contain many different types of geometry however, the associated attributes must have the same number and type of fields for each entity in the file.

The MapInfo reader can open datasets in which the attributes are stored as .dat, .dbf, .mdb, .accdb, or .xls files. In other words, the MapInfo reader supports datasets of Type NATIVE, Type DBF, Type ACCESS and Type XLS.

The number and type of attributes associated with each entity is specified by the user. There must be at least one attribute field defined before a MapInfo file can be created.

The following illustration shows a MapInfo file containing three region entities. Note that the second polygon contains a hole while the third polygon is an aggregate of two disjoint polygons, one of which contains a hole. Each geometric entity in turn corresponds with one record in the attribute table.


10

MapInfo TAB Quick Facts

Format Type Identifier MAPINFO MITAB

Dataset Type Directory or File Feature Type File base name Typical File Extensions .tab (.dat, .id, .map, .ind) User-Defined Attributes Yes Coordinate System Support Yes

Geometry Support Geometry Supported? Geometry Supported? aggregate yes point yes circles yes polygon yes circular arc yes raster no donut polygon yes solid no

elliptical arc yes surface no

ellipses yes text yes line yes z values no none yes

OS (GB) MasterMap

OS MasterMap Overview OS MasterMap® provides a highly detailed view of Great Britain's landscape including, among other things, individual buildings, roads and areas of land. In total, it contains in excess of 400 million individual features.

OS MasterMap is stored in an GML based format. More information on data products available in the OS MasterMap program can be found on their website.

OS MasterMap Quick Facts

Format Type Identifier GML Dataset Type File

Feature Type Varies depending on the application schema

Typical File Extensions .gml, .xml User-Defined Attributes Yes Coordinate System Support Yes


11

Geometry Support

Geometry Supported? Geometry Supported? aggregate yes point yes circles no polygon yes circular arc yes raster no donut polygon yes solid no


ellipses no text no line yes z values yes none yes

Spatial Data Transfer Standard (SDTS)

SDTS Overview The SDTS reader reads data structured according to the SDTS Topological Vector Profile (TVP). This format was used in the 90s for distribution of data by the USGS, and several datasets are still available online in this format. In modern times, GML based data exchange is more likely to be used.

More information on SDTS can be found at: http://mcmcweb.er.usgs.gov/sdts

SDTS Quick Facts

Format Type Identifier SDTS Dataset Type File Feature Type Feature role Typical File Extensions .ddf User-Defined Attributes No Coordinate System Support Yes

Geometry Support Geometry Supported? Geometry Supported? aggregate no point yes circles no polygon yes circular arc no raster no donut polygon yes solid no



12

Geometry Support Geometry Supported? Geometry Supported? ellipses no text no line yes z values yes none yes

Vector Product Format (VPF) Coverage

VPF Overview The Vector Product Format (VPF™) is a standard format, structure, and organization for large geographic databases. VPF data is stored in a structure described in the Military Standard, Vector Product Format, MIL-STD-2407. The Standard specifies the structure for directories, tables, table columns, table join relationships, and media exchange conventions for all VPF data. While the Standard describes the structure, it does not describe the contents of a set of VPF data; this is the role of "VPF Product Specifications."

The VPF Reader module provides the Feature Manipulation engine (FME) with access to data in any of the number of formats that follow the VPF specification. This includes, but is not limited to, data that adheres to the Digital Chart of the World (DCW), Digital Nautical Chart (DNC), VMap Level 0, VMap Level 1, VMap Level 2, and UVMap database standards.

VPF Reader Quick Facts

Format Type Identifier VPF, VPF_DB (see note*) Dataset Type Directory or File

Feature Type Feature class name Metadata table name

Typical File Extensions *.*ft, dht (see note*) User-Defined Attributes Yes Coordinate System Support No

Geometry Support Geometry Supported? Geometry Supported? aggregate no point yes circles no polygon yes circular arc no raster donut polygon yes solid no


ellipses no text yes line yes z values no


13

Geometry Support Geometry Supported? Geometry Supported? none no

For More Information A wealth of additional information on the data formats and capabilities of the free FME FDO provider is available online. In addition to the many links found in this document, the links below may be useful:

http://www.safe.com/solutions/for-applications/autodesk/autodesk-autocad-map-3d/fme-fdo-provider/ -- the information and download for the free FME FDO Providder

http://www.safe.com/autodesk - information about Safe’s support for Autodesk products

http://www.safe.com/fmefdo - overview and general information about Safe’s FDO implementation

http://www.fmepedia.com - troubleshooting information about all FME formats and capabilities

[email protected] - my email address

@DaleAtSafe - my twitter handle.

Thanks so much for your interest and support, and for making it to the bottom of this document! If you got here, and would like a free “CADfused and GISorganized” T-Shirt, I’ll send one out to the first 3 people who email me with their shipping address and T-shirt size! Thanks again!

Documents

Accessing More CAD and GIS Data within AutoCAD® …aucache.autodesk.com/au2011/sessions/5176/class_handouts/v6_v3_GS...Accessing More CAD and GIS Data within AutoCAD® Map 3D 2 of