Load OSM File - missing features

Nov 26, 2010 at 1:49 PM

Hey, I know that you're about to release the 1.1 Beta soon, but today I noted another issue that you might be interested in (unless of course you already know about it).

I downloaded some OSM raw data, updated it and extracted an area using Osmosis. I then loaded the data into ArcMap via the 'Load OSM File'-Tool. On the first glance all that went without any problems, but after a while it turned out that some features (Points and Lines - Polygons I haven't checked) just didn't make it into ArcMap - it appears they simply were not converted to the feature class as they don't even exist in the attribute table. I checked the OSM XML-File that I used, and I found them in there without any problem. I also checked by downloading the relevant area directly with the 'Download OSM Data'-Tool and that also turned out to be fine. Do you know anything about this problem?

I also was wondering about the implementation of Multipart Polygons using the extension. Is there a way to extract the information that is stored in the blobs of the relation table so I can assign it to the relevant features? For example, take a building that is tagged via a relation describing the inner and outer polygon and its other attributes; in ArcMap these polygons don't contain any information, which makes it pretty hard to symbolise them when using the features attributes. Is this a recognized problem or am I simply missing something?

Developer
Nov 29, 2010 at 5:22 PM

Lainon,

what you are describing could be related to issue #7248. I believe the fixes went into a very early 1.1 build (http://esriosmeditor.codeplex.com/releases/view/53087) In case that version doesn't fix the issue, would it be possible to share the raw OSM file?

Multipart features is the main topic for the 1.1 release. Essentially the tool will translate homogenous multipart relations (meaning containing only polygons or only lines) into the appropriately matching geometry. Super-relations and heterogenous relations are skipped and stored in the relations table.

- Thomas

Nov 30, 2010 at 9:54 AM

Hey,

I just installed the 1.1 Beta and loaded the OSM file into ArcMap - the problem still appears. It actually does sound a bit like issue #7248, as the nodes of the line feature load just fine and seem like dummy points that way.

The way in question is Way Weiße Elster (4357978); it is part of the Relation Weiße Elster (387339) and I thought that maybe the relation was causing this behavior, but when downloading the area via Download OSM Data (as I mentioned before) everything's fine. Other parts of the relation are loaded without problem as well (via Load OSM File).

The XML is about 11MB - how can I get it to you?

Karl

Developer
Nov 30, 2010 at 3:02 PM

Karl,

you might want to try to email the file directly to me. My email is temge at esri.com

-Thomas

Developer
Dec 1, 2010 at 5:01 PM

Karl,

I just looked at the file you've sent me and the tools work as designed.

The "issue" here is that the way 4357978 is referencing a node (767737967) which is not part of the osm file. At that point the import tool "flags" the way as not reproduceable (with the given information) and discards the whole way. The relation information is still intact though as other ways (which are part of that relation) still show up. Touching those ways would change the relation itself, hence the relation information needs to stay around.

By the way, this behavior will not change in 1.1 and seems to be an "issue" with OSM extract files that they are not referential inclusive. I kind of understand the reasoning behind it (limiting the size of the download) but those are some of the consequences. For smaller extents like yours I would recommend to use Download OSM tool which has the option to resolve any referential issue. The Load OSM File tool is kind of intended to deal with larger extracts like the US states or countries like Haiti.

Thanks for the feedback.

- Thomas

 

Dec 2, 2010 at 2:35 PM

Hey,

thank you for looking into this and for the explanation! I'm aware that the Load OSM Tool is designed for large datasets; but as the Download OSM Data Tool limits the number of downloadable nodes to 50000, even when working on just one city its not usable - plus I didn't want to send you a mail with a 500MB file attached to it...

I still have another issue - sorry to give you so much trouble, but I'm using the extension quite a lot at the moment - that struck me when I compared the files earlier on. The Load OSM Tool (as opposed to the Download OSM Data Tool) seems to have difficulties with closed ways (with identical start and end nodes) that are not polygons, like roundabouts or streets that reconnect to themselves at some point (Way 84843148 or 19246252 ). The part of the way between the last and the first node (or any other of the way it should connect to) simply doesn't show up. When downloading the area though everything's as it should be. If you want a screenshot or the files I'd be happy to mail them to you.

Karl

Developer
Dec 2, 2010 at 6:24 PM

Karl,

the ways you've mentioned do show up in my current build of the tools - and I would really like you test the new 1.1 build as soon as possible  - when I actually do have a public build ready. However you do bring up an issue I haven't considered yet. Are there self-intersecting ways in OSM? As I am currently handling multi-part features those ways could eventually be turned into a relation when it goes through the ESRI editing chain due to the self-intersecting lines be split into multiple parts.

Do know of such a case (a single way crossing and continuing over itself)?

- Thomas

 

Dec 3, 2010 at 2:11 PM

Hey,

fair enough - I'll wait for the public version of the 1.1 build and get back to you then.

As far as I know there are self-intersecting ways in OSM, but they are actually not supposed to exist and usually being errors. There is a page in the Wiki with links to that kind of ways.

Karl