Load OSM File error

Dec 9, 2011 at 8:47 AM

Hi every one,

I have started using ArcGIS Editor for OSM from stable version 1.1. Everything was OK, i have converted lot of OSM files with out any problem till yesterday.

Since yesterday i am getting this error message.

Der Objektverweis wurde nicht auf eine Objektinstanz festgelegt.
   bei ESRI.ArcGIS.OSM.GeoProcessing.OSMGPFileLoader.Execute(IArray paramvalues, ITrackCancel TrackCancel, IGPEnvironmentManager envMgr, IGPMessages message)
Failed to execute (OSMGPFileLoader).

Translation is

Object reference not set to an object instance.
    at ESRI.ArcGIS.OSM.GeoProcessing.OSMGPFileLoader.Execute (IArray ParamValues , ITrackCancel TrackCancel, IGPEnvironmentManager envMgr, IGPMessages message)
Failed to execute (OSMGPFileLoader).

I have even tried ArcGIS Editor for OSM 2 (beta) version but  still getting same error message.

Strange thing is every thing was working before but now, all of sudden some thing goes wrong.

I am local Administrator of the machine and other working environment are as follows

OS: Windows Server 2008 R2 (German) 64 Bit.

ArcGIS 10 (Editor) Service pack 3

Developer
Dec 9, 2011 at 6:02 PM

Is there a way to get a subset of the OSM file or a download location for the file you are attempting to load?

- Thomas

 

Dec 12, 2011 at 7:12 AM

hi Thomas,

I have used OSM from here and here

One another thing, i have used my old OSM as well but my old OSM is also not working. That means there is nothing wrong with OSM data.

Can Service pack3 be a problem?

iRfAn

Developer
Dec 12, 2011 at 5:58 PM

Could be. Currently I am testing the import of Germany and the full planet file in various OS and DB configurations and so far I have not seen an issue with SP2. Once the current loading is complete I'll add SP3 and test again.

- Thomas

Developer
Dec 14, 2011 at 9:22 PM

Irfan,

at what point in the loading process are you seeing this error message? After loading the points, ways or relations? I have currently 10 millions points loaded and it is still going strong.

- Thomas

 

Dec 15, 2011 at 8:01 AM
Edited Dec 15, 2011 at 2:36 PM

hi Thomas,

I am getting this message exactly at the start, you can say i will get the message in first 2-4 seconds with out any loading.

iRfAn

Dec 26, 2011 at 7:27 PM
Edited Dec 26, 2011 at 7:27 PM

Having the same issues as well.

Developer
Dec 27, 2011 at 12:51 AM

Hmm, not sure why this is happening.

Can you confirm that you have 2 xml files in the bin directory? What part of the sortware did you install? Desktop, desktop and server, or server only? What OS are you running?

- Thomas

Dec 27, 2011 at 2:15 PM

I installed only the desktop portion and I'm running Windows 7 Professional X64. I'll have to check this evening concerning the .bin files. And for a little more background, I've never used the 1.x versions. This is my first time using the editor, and with ArcGIS Desktop SP3. Never used it prior to SP3. One thing I have seen though is that it seems to work with a very small OSM dataset. When I say small, I mean a road or two. Seems like anything larger causes the error. I'll see if I can duplicate it this evening.

Developer
Dec 31, 2011 at 6:43 AM

Your issue unfortunately doesn't sound related to what Irfan seeing. So far we have tested various OSes with different language settings and various osm file sizes (from Lichtenstein, Croatia, Germany, and Europe) and we were unable to reproduce the issue.

- Thomas

Jan 31, 2012 at 10:25 AM

Hi,

I need help for the same topic, another error:

 

Wird ausgeführt: OSMGPFileLoader C:\Arbeit\OSMEditor\G662120125.osm CONSERVE_MEMORY C:\Arbeit\OSMEditor\G662120125.gdb\LoadOSM C:\Arbeit\OSMEditor\G662120125.gdb\LoadOSM\LoadOSM_osm_pt C:\Arbeit\OSMEditor\G662120125.gdb\LoadOSM\LoadOSM_osm_ln C:\Arbeit\OSMEditor\G662120125.gdb\LoadOSM\LoadOSM_osm_ply
Startzeit: Tue Jan 31 11:18:32 2012
Preparing geodatabase...

[snip 1583892 nodes loaded...
261481 ways loaded...]

Resolving multi-part geometries...
1895 relations loaded...
Wird ausgeführt: AddIndex GPT0 OSMID osmID_IDX NON_UNIQUE NON_ASCENDING
Startzeit: Tue Jan 31 11:59:31 2012
Erfolgreich bei Tue Jan 31 11:59:32 2012 (Verstrichene Zeit: 1,00 Sekunden)
Wird ausgeführt: AddIndex GPT0 osmoldid;osmnewid osmID_IDX NON_UNIQUE NON_ASCENDING
Startzeit: Tue Jan 31 11:59:32 2012
Erfolgreich bei Tue Jan 31 11:59:32 2012 (Verstrichene Zeit: 0,00 Sekunden)
Update remaining references...
Workspace oder Datenquelle ist schreibgeschütz. [Der C:\Arbeit\OSMEditor\G662120125.gdb-Workspace ist schreibgeschützt.]
   bei ESRI.ArcGIS.Geodatabase.IFeatureClass.Update(IQueryFilter filter, Boolean Recycling)

   bei ESRI.ArcGIS.OSM.GeoProcessing.OSMGPFileLoader.Execute(IArray paramvalues, ITrackCancel TrackCancel, IGPEnvironmentManager envMgr, IGPMessages message)
Fehler beim Ausführen von (OSMGPFileLoader).
Fehlgeschlagen bei Tue Jan 31 11:59:34 2012 (Verstrichene Zeit: 41 Minuten 2 Sekunden)

(schreibgeschützt = write protected)

Small OSM files load without error.

Am I running into some limitation of file databases? Is Windows XP 32 bit too limited for this Task?

Thanks, Bonner

Developer
Jan 31, 2012 at 3:17 PM

Bonner,

not sure why you would run into a lock. I believe that the German translation of the message is misleading - write protected vs. geodatabase lock. Is only this file giving you problems or do you continuously get this error once your OSM file is beyond a certain number of features? If possible I would like to get a copy of the osm and test it on my machine. What version of the tool are you running?

Millions of features is not problem for the geodatabase and before Christmas I did a number of loading testing with a full Germany extract and the loading completed successfully. The file loader geoprocessing tool does not require a powerful machine and XP 32bit will do just fine.

Thanks,

Thomas

 

Jan 31, 2012 at 7:44 PM
Edited Jan 31, 2012 at 7:44 PM

Hi Thomas,

I used data downloaded from geofabrik.de for Niedersachsen (Lower Saxony), tried the whole file and small areas cut with osmosis.

Now I made a series of tests with data for Berlin, also downloaded from geofabrik.de. I narrowed the size down until I got a successful import. That file gets generated with

osmosis --rb berlin.osm.pbf --bb left=13.41 top=52.51 right=13.43 bottom=52.5 --wx berlin6561.osm

file size 2 MB; 6561 Nodes reported

I then started this import ten times into different feature datasets in the same file database. Result: Failed, Success, Crash, Failed, Failed, Crash, Success, Failed, Success, Success.

Success: The import was visible on the map, event log shows a bunch of warnings about repaired geometries. Failed: Error as shown above. Crash: Message from Microsoft Visual C++ Runtime Library:

 

Runtime Error!
Program C:\Programme\ArcGIS\Desktop 10.0\Bin\ArcMap.exe
R6025
- pure virtual function call

then ArcMap was gone.

I also did some tests with login as admin, using ArcCatalog, in ArcMap use named file vs. unnamed new file, with no success.

I am using Arc Editor 10.0 with SP 3 installed, OSM Editor 1.1 - stable version from the download page.

Please let me know if you need additional information or files.

Thanks, Bonner

Feb 1, 2012 at 7:05 AM
Edited Feb 1, 2012 at 7:06 AM

Switched to 2.0 beta this morning and repeated the same test 10 times. The tool is running faster now.

Result: Failed, Failed, Failed, Failed, Failed, Success, Failed, Success, Success, Crash

Crash without any message this time, ArcMap window just disappeared - I had to logout from Windows to delete the file geodatabase.

Attached are logs from another fail and success:

Meldungen
Wird ausgef�hrt: OSMGPFileLoader C:\Arbeit\OSMEditor\Berlin6561.osm CONSERVE_MEMORY C:\Arbeit\OSMEditor\Berlin.gdb\Test01 C:\Arbeit\OSMEditor\Berlin.gdb\Test01\Test01_osm_pt C:\Arbeit\OSMEditor\Berlin.gdb\Test01\Test01_osm_ln C:\Arbeit\OSMEditor\Berlin.gdb\Test01\Test01_osm_ply
Startzeit: Wed Feb 01 08:53:24 2012
Counting elements in OSM file...
Counted 6561 nodes, 1435 ways, and 46 relations.
Preparing geodatabase...
6561 nodes loaded...
Building point indexes...
Wird ausgef�hrt: AddIndex C:\Arbeit\OSMEditor\Berlin.gdb\Test01\Test01_osm_pt OSMID osmID_IDX UNIQUE NON_ASCENDING
Startzeit: Wed Feb 01 08:53:34 2012
Erfolgreich bei Wed Feb 01 08:53:35 2012 (Verstrichene Zeit: 1,00 Sekunden)
1290 ways loaded...
Building line indexes...
Wird ausgef�hrt: AddIndex C:\Arbeit\OSMEditor\Berlin.gdb\Test01\Test01_osm_ln OSMID osmID_IDX UNIQUE NON_ASCENDING
Startzeit: Wed Feb 01 08:53:40 2012
Erfolgreich bei Wed Feb 01 08:53:40 2012 (Verstrichene Zeit: 0,00 Sekunden)
Building polygon indexes...
Wird ausgef�hrt: AddIndex C:\Arbeit\OSMEditor\Berlin.gdb\Test01\Test01_osm_ply OSMID osmID_IDX UNIQUE NON_ASCENDING
Startzeit: Wed Feb 01 08:53:40 2012
Erfolgreich bei Wed Feb 01 08:53:40 2012 (Verstrichene Zeit: 0,00 Sekunden)
Resolving multi-part geometries...
33 relations loaded...
Wird ausgef�hrt: AddIndex GPT0 OSMID osmID_IDX UNIQUE NON_ASCENDING
Startzeit: Wed Feb 01 08:53:47 2012
Erfolgreich bei Wed Feb 01 08:53:47 2012 (Verstrichene Zeit: 0,00 Sekunden)
Wird ausgef�hrt: AddIndex GPT0 osmoldid;osmnewid osmID_IDX NON_UNIQUE NON_ASCENDING
Startzeit: Wed Feb 01 08:53:47 2012
Erfolgreich bei Wed Feb 01 08:53:47 2012 (Verstrichene Zeit: 0,00 Sekunden)
Updating remaining references...
Workspace oder Datenquelle ist schreibgesch�tz. [Der C:\Arbeit\OSMEditor\Berlin.gdb-Workspace ist schreibgesch�tzt.]
   bei ESRI.ArcGIS.Geodatabase.IFeature.Store()

   bei ESRI.ArcGIS.OSM.GeoProcessing.OSMGPFileLoader.Execute(IArray paramvalues, ITrackCancel TrackCancel, IGPEnvironmentManager envMgr, IGPMessages message)
Fehler beim Ausf�hren von (OSMGPFileLoader).
Fehlgeschlagen bei Wed Feb 01 08:53:49 2012 (Verstrichene Zeit: 25,00 Sekunden)

 

Meldungen
Wird ausgef�hrt: OSMGPFileLoader C:\Arbeit\OSMEditor\Berlin6561.osm CONSERVE_MEMORY C:\Arbeit\OSMEditor\Berlin.gdb\Test02 C:\Arbeit\OSMEditor\Berlin.gdb\Test02\Test02_osm_pt C:\Arbeit\OSMEditor\Berlin.gdb\Test02\Test02_osm_ln C:\Arbeit\OSMEditor\Berlin.gdb\Test02\Test02_osm_ply
Startzeit: Wed Feb 01 08:54:08 2012
Counting elements in OSM file...
Counted 6561 nodes, 1435 ways, and 46 relations.
Preparing geodatabase...
6561 nodes loaded...
Building point indexes...
Wird ausgef�hrt: AddIndex C:\Arbeit\OSMEditor\Berlin.gdb\Test02\Test02_osm_pt OSMID osmID_IDX UNIQUE NON_ASCENDING
Startzeit: Wed Feb 01 08:54:15 2012
Erfolgreich bei Wed Feb 01 08:54:16 2012 (Verstrichene Zeit: 1,00 Sekunden)
1289 ways loaded...
Building line indexes...
Wird ausgef�hrt: AddIndex C:\Arbeit\OSMEditor\Berlin.gdb\Test02\Test02_osm_ln OSMID osmID_IDX UNIQUE NON_ASCENDING
Startzeit: Wed Feb 01 08:54:21 2012
Erfolgreich bei Wed Feb 01 08:54:21 2012 (Verstrichene Zeit: 0,00 Sekunden)
Building polygon indexes...
Wird ausgef�hrt: AddIndex C:\Arbeit\OSMEditor\Berlin.gdb\Test02\Test02_osm_ply OSMID osmID_IDX UNIQUE NON_ASCENDING
Startzeit: Wed Feb 01 08:54:21 2012
Erfolgreich bei Wed Feb 01 08:54:21 2012 (Verstrichene Zeit: 0,00 Sekunden)
Resolving multi-part geometries...
34 relations loaded...
Wird ausgef�hrt: AddIndex GPT0 OSMID osmID_IDX UNIQUE NON_ASCENDING
Startzeit: Wed Feb 01 08:54:27 2012
Erfolgreich bei Wed Feb 01 08:54:28 2012 (Verstrichene Zeit: 1,00 Sekunden)
Wird ausgef�hrt: AddIndex GPT0 osmoldid;osmnewid osmID_IDX NON_UNIQUE NON_ASCENDING
Startzeit: Wed Feb 01 08:54:28 2012
Erfolgreich bei Wed Feb 01 08:54:28 2012 (Verstrichene Zeit: 0,00 Sekunden)
Updating remaining references...
Wird ausgef�hrt: RepairGeometry C:\Arbeit\OSMEditor\Berlin.gdb\Test02\Test02_osm_ln DELETE_NULL
Startzeit: Wed Feb 01 08:54:34 2012
WARNING 000986: C:\DOKUME~1\Work\LOKALE~1\Temp\Test02_osm_ln1.txt enth�lt eine vollst�ndige Liste der nicht einfachen Features.
WARNING 000107: Feature 343 wurde aufgrund leerer Geometrie gel�scht.
Erfolgreich bei Wed Feb 01 08:54:34 2012 (Verstrichene Zeit: 0,00 Sekunden)
Wird ausgef�hrt: RepairGeometry C:\Arbeit\OSMEditor\Berlin.gdb\Test02\Test02_osm_ply DELETE_NULL
Startzeit: Wed Feb 01 08:54:34 2012
WARNING 000986: C:\DOKUME~1\Work\LOKALE~1\Temp\Test02_osm_ply1.txt enth�lt eine vollst�ndige Liste der nicht einfachen Features.
WARNING 000461: Feature 15 wurde aufgrund von falsche Ringanordnung repariert.
[... snipped ...]
WARNING 000461: Feature 86 wurde aufgrund von falsche Ringanordnung repariert.
Erfolgreich bei Wed Feb 01 08:54:40 2012 (Verstrichene Zeit: 6,00 Sekunden)
Erfolgreich bei Wed Feb 01 08:54:40 2012 (Verstrichene Zeit: 32,00 Sekunden)

Thanks, Bonner

Feb 1, 2012 at 9:52 AM

Notice the different numbers in "ways loaded" and "relations loaded". It's definitely the same osm-file!

Developer
Feb 1, 2012 at 3:28 PM
Edited Feb 1, 2012 at 3:39 PM

I'll take a look today and keep you posted.

The difference in numbers is explained as the osm files do not guarantee completeness. Meaning that a way could be listed in the osm file but not all nodes that are referenced by the way are acutally in the file itself. The same is true for relations. In case the tool encounters missing information then this feature (line or polygon -- or in OSM speak way or relation) is not loaded because we are not able to acurately reproduce it.

The number of points/nodes should always be the same and the number of ways and relations may or may not be the same.

- Thomas

 

Feb 1, 2012 at 4:41 PM

Thanks you temge!

I don't agree that the number of ways and relations may not be the same. Both runs read the same, unchanged osm file, so why should information be missing only once?

Here are the files:

Berlin6561.osm

Berlin.gdb

Berlin.gdb contains the two feature sets that were generated by the failed and the successed run with logs posted above. I found three features that are in dataset test01 (which is expected to be corrupt), but not in dataset test02:

ln OBJECTID 343 OSMID Relation 443545

This one is not drawn by ArcMap, it should be a polygon and duplicates the polygons with OBJECTID 768 to 770.

ply OBJECTID 889 OSMID Way 105066740

This one is inner part of multipolygon:

ply OBJECTID 960 OSMID Relation 1490786

Developer
Feb 2, 2012 at 4:50 AM

Bonner,

I was able to reproduce the crashes and made some code changes. After the code changes I was able to run the tool inside a model builder loop 10 times in a row executing successfully. We will have a new beta version to test by the end of the week the latest. Please download the next beta and confirm that it is working in your environment as well.

Why the tool is reporting different numbers of loaded ways and relation is beyond me. I was unable to reproduce the different results on my machine. All of the ten loads were reporting 1290 ways and 33 relations.

Please retest with the new version and let us know if there are still issues.

- Thomas

Feb 2, 2012 at 6:25 AM
and confirm that it is working in your environment as well.

Wow! I hope so. Thank you very much!

Feb 7, 2012 at 3:16 PM

Hi Thomas, Hi all,

sorry, I get the same errors using Beta 3. At least I had no more ArcMap crashes in my test series. :-)

In ten runs using the same Berlin6561.osm I got these results:

  • Test #2 ran successful, 1290 ways and 33 relations loaded. (so it seems success is independent from number of ways loaded)
  • Test #1 stopped with error, 1289 ways and 33 relations loaded. The missing way is OSM ID 37846120 - though I think this is no help.
  • Test #3 to #10 stopped with error, 1290 ways and 33 relations loaded.

I disabled background geo processing, and now get more detailed progress messages than that in the result window. Tests #4 and #6 showed "Updating way relations ..." as last message, and the error came from ESRI.ArcGIS.Geodatabase.IFeatureClass.Update(). The other failed tests stopped with "Updating point reference counts ..." with error from ESRI.ArcGIS.Geodatabase.IFeature.Store().

I wonder why I am the only one complaining about this. Maybe I get another trial of ArcGIS and install it on another machine. Unfortunately I currently don't have access to an up-to-date Windows 7 desktop.

Thanks for your help, Bonner

Developer
Feb 7, 2012 at 4:09 PM

Bonner,

I'll set some more time aside for testing today. Just to recap: you are running Arc Editor 10 with SP3 on Windows XP 32bit with SP3 and German language settings, correct?

When you are loading the Berlin6561.osm multiple times are you always loading it into the same feature class? How are you loading the data, through a Python script, using ModelBuilder or the tool UI?

Thanks,

Thomas

 

Feb 7, 2012 at 4:49 PM
Edited Feb 7, 2012 at 5:51 PM

Hi Thomas,

system is correct. I load into a new feature class dataset each time, located in the same gdb. I use the catalog window to start "Load OSM File", and fill the form. I used neither Python nor ModelBuilder, but I will try them now.

Edit: Tried ModelBuilder and Python, got the same error.

Bonner

Feb 8, 2012 at 4:02 AM
Edited Feb 8, 2012 at 4:31 PM

I am not sure now, but the culprit may be Microsoft Security Essentials Extensions with realtime scanning.

Developer
Feb 8, 2012 at 5:17 AM

I just installed Microsoft Security Essentials and with real-time scanning enabled I am seeing errors and funny numbers. Wow, we'll mark it as a known issue.

 

Feb 8, 2012 at 4:47 PM

Thank you Thomas for confirming this!

MSE scanning can be restricted by path/filename, by file extension or per executable. I excluded ArcCatalog.exe, ArcMap.exe, ArcSOCP.exe and ArcSOMP.exe. I now loaded Niedersachsen.osm successfully (about 2 GB osm file and 8 hours loading time).

Sad that this took so much of your and my time. I should have thought on antivirus earlier. Thanks again for your support!

Bonner

Developer
Feb 8, 2012 at 5:08 PM

It is actually a known issue and after talking to our product specialist he pointed me to a published Knowledge Base article at http://support.esri.com/en/knowledgebase/techarticles/detail/35504 

Feb 9, 2012 at 11:13 PM

Hi there,

I got the same error message if i try to open an osm file:

Der Objektverweis wurde nicht auf eine Objektinstanz festgelegt.
   bei ESRI.ArcGIS.OSM.GeoProcessing.OSMGPFileLoader.Execute(IArray paramvalues, ITrackCancel TrackCancel, IGPEnvironmentManager envMgr, IGPMessages message)
Failed to execute (OSMGPFileLoader).

The Size dosn't matter, the whole File for Niedersachsen takes just a few seconds more than an little 2MB map. The strange thing is i don't got any ativirus-software installed yet.  I set up a new Windows 7 64 Bit today. After a few tests I did install MSE and used Bonners configurations, but it doesn't help.

I'm working with ArcGis 10.3 (English)

Does somebody got any ideas?

Roland

Feb 10, 2012 at 6:33 AM

Hi Roland,

Ouch! :-) For all who are reading this: Installing MSE is never a solution for ArcGIS related problems, probably you will get new trouble. It . I hope you don't run two antivirus programs in parallel now?

This error message is the same that irfan_latif saw in the first post. I see no solution yet in this thread. As far as I see it is not related to the errors I had.

Good luck, Bonner

Developer
Feb 10, 2012 at 4:20 PM

Roland,

I agree with Bonner. If the loading process fails within a couple of seconds then that is because the tool can't find some configuration files.

The code should fail a little more gracefully than with a null pointer exception. Did you install desktop and server or only desktop?

- Thomas

Feb 11, 2012 at 9:45 AM
Edited Feb 15, 2012 at 10:37 PM

Hello Thomas and Bonner,

thanks for your answers. I'm using ArcInfo Desktop 10.3 (no Server installed), OSM Editor 2.0 beta. This runs on a Win7 SP1 System. I have got administrative rights and no (more) anti-virus Software is installed. Hardware is normal up to date standard. I have to say, I am a complete newbe in ArcGis, I just started to work with it for my BA-thesis. Is it possible to get a more detailed error-report from ArcGis?

I have installed the same ArcGis System on a 32Bit WinXP-System. If I ad the OSM Editor toolbox an use the „open OSM“ tool, I get the same error. But I can open small OSm-files using the „add data“ dialog. On my Win7 system OSM-Files aren’t shown in the “add data” dialog. Larger Files crash on WinXP because the hard disk drive is to small…

Roland 

 

Update: I have solved my problem and it was my fault. I have not seen the hint, that I have to safe the file into a geodatabase... I'm sorry that I wasted your time.