Recently, I ran across a problem that I didn’t think should be there – repeated “Out of Memory” messages when trying to import an ESRI SHP file into Civil 3D 2009. The really bad part about the issue was the fact that I could import the same SHP file into Civil 3D 2008, Civil 3D 2007, and Civil 3D 2006 with no problem. For the purposes of this scenario, FDO was not an option as the file had to be available to users in the field with mobile computers. So how was I to get the entire file imported and saved in a drawing without running into the error? Simple – FDO was the answer after all!
The computer being used for this testing was my laptop with decent specs – a Core 2 Duo processor running at 2.8 GHz, 4 GB of installed RAM, an ATI Mobility FireGL V5700 video card with 512 MB of dedicated RAM, and Windows XP x32 without the /3GB switch enabled. I was trying to import the SHP file which had a combined size of just over 136.8 MB into a blank drawing using the MAPIMPORT command. The command processed part of the file and then stopped and gave me a warning that I was running low on system memory, and the command aborted. This left me with only a partial file loaded into my drawing, which was unacceptable.
I decided to try this on my desktop since I knew it had better specs – a Core 2 Duo processor running at 3.0 GHz, 6 GB of installed RAM, an nVidia Quadro FX 1700 video card with 512 MB of dedicated RAM and access to up to 2 GB of system RAM, and Windows Vista x64. The same exact SHP file imported perfectly to a blank drawing on this machine, mainly due to the added available RAM. At this point, I had a definitive answer – just upgrade the computers. Unfortunately, that is not always a viable solution, and most definitely wasn’t in this particular case.
After thinking about it for a while, banging my head a bit, and throwing my mouse across the office a few times, I decided to ping my good friend Doug Lauritsen from the geospatial product support team. Doug told me a bit of bad news, then totally redeemed himself by giving me an incredible workaround.
The bad news: the code base for Map 3D has grown significantly in past releases. This means that what might have been possible in a prior release might not be possible in a current release because the current release simply has a larger footprint with regards to system resources. That’s a fancy way of saying that Map 2009 just needs a bit more head room to process the same task.
Then he prefaced the good news with a tidbit of information that I wasn’t aware of – some geospatial file formats are more efficient than others. Specifically, the Autodesk SDF (Spatial Data Format) file format is more efficient than the ESRI SHP converter. Note that this only pertains to AutoCAD Map 3D based products – I am making no assertions that the SDF file is superior to SHP, just that it’s more efficient in Map 3D. So he thought that if I had a SDF file containing the same data as the SHP file, then I would likely be able to import that.
Here’s where FDO comes in – since FDO (Feature Data Objects) is a connection rather than an import, it requires less overhead. FDO isn’t trying to take the date from one file and convert it to a format that AutoCAD can utilize, it is simply displaying the file and associated data in the native ESRI format. I could connect to the suspect SHP file and display it very easily on the Windows XP x32 machine, but I had to take it a step further – I had to get that data over into a different format. Since I had connected to the SHP file, I had a corresponding data layer listed in the Display Manager of my Map Workspace. If I right click that data layer, I have an option to Export Layer Data to SDF. Selecting that option creates a nice new SDF file coming in at a nice svelte 120.6 MB – something looks a bit more efficient already!
Once the SDF file is created, you can come full circle – back to the blank drawing I went, and back to the MAPIMPORT command…changing file formats, selecting the new SDF file, and wait with much anticipation.
IT WORKED! Even better, there wasn’t much hesitation at all. This gets the data fully imported into the drawing – linework complete with associated data records. The drawing could be saved and the field users were happy.
Give this a try next time you have a large SHP file – you’ll be happy you avoided any nasty messages and your drawing will have everything you need in it.
Have fun!
Subscribe
Twitter