Spanish Cadastre/Buildings Import/Data Conversion/Software/Specifications

	Import guide		Projects management		Results		Documentation
Specifications describes the solutions to some of the problems faced by the software developed for this Buildings Import. Data transformation Algorithm to split multipart buildings Motivation: buildings and other datasets with multipart geometries. Steps: Iterates over each building. If their geometry is of type WKBMultiPolygon: A new building is added for each polygon copying the original fields. Deletes the old building. From this moment, the field 'localId' will not be any more a unique identifier for buildings. Implementation: layer.PolygonLayer.explode_multi_parts Building parts outside the building footprint Motivation: To delete building parts below ground level and building parts outside the building footprint, create footprint for parts without associated building. Steps: Select parts with 'numberOfFloorsAboveGround' = 0. Delete them. For each feature in this layer: If it is a building part: If it have levels under ground ('numberOfFloorsBelowGround' < 0) but not above ground ('numberOfFloorsAboveGround' = 0), drops it. If it have an associated building and is not inside it, drops it. It have associated building if there exists a building with the value of the catastral reference in the 'base:localId' field. If it haven't an associated building, adds it to a dictionary grouped by the cadastral reference. With this dictionary of parts without associated building, for each building cadastral reference, generates the building footprint from the merging of all the parts. Implementation: layer.ConsLayer.remove_outside_parts Reduction of building parts Motivación: Reduction of building parts. Steps For each building and its parts (matches their cadastral references). Calculates the minimun and maximum values of levels under and above ground in the building parts. Assigns these values to the building footprint. For each pair of diferent levels under and above ground values in the parts of the building and for each set of parts with this values. If these values are equal to the values calculated for the building footprint, delete the parts. Otherwise, merge the adyacent parts. Implementation:: layer.ConsLayer.merge_building_parts Detection of swimming pools inside buildings Motivación: Detection of swimming pools inside buildings. Steps For each building and its parts (matches their cadastral references). For each swimming pool in the same parcel (matches their cadastral references). If the pool is inside a building assign to it the tag layer=1. If the building footprint is coincident with the pool deletes the building. Otherwise, Deletes the inner rings of the building geometry that are coincent with the pool. Deletes the parts of the building that are coincident with the pool. Implementation:: layer.ConsLayer.merge_building_parts Invalid geometries Motivation: To delete junk geometries and vertices with too low angle. Parameters: 'acute_inv' = 5° 'min_area' = 0.05 m2 'dist_inv' = 10 cm 'straight_thr' = 2° Steps: For each geometry 'geom' in this layer. For each ring with index 'i' in the geometry. For each vertex 'v' in the ring. If the angle of the vertex is acute ('angle_v' < 'acute_inv'). Try to delete the vertex. If the resulting geometry is not valid or its area is less than 'min_area'. If this is the outermost ring (i == 0) the geometry is invalid, drops it. If this is a inner ring (i > 0) deletes the ring from the geometry. Otherwise and if the ring have more than four vertices. Take the 'angle_a' to the nearest adyacent vertex ('va'). Take the distance 'c' from 'va' to the segment from 'v' to the farther adyacent vertex ('vb'). If 'va' is acute ('angle_a' < 'acute_inv') and it pass a filter ('c' < 'dist_inv'), 'v' and 'va' forms a zig-zag, delete them. If 'va' is not straight (|180 - angle_a| < 'straight_thr') and pass a filter ('c' < 'dist_inv') this is a strike vertex. Take the 'vx' point formed by the projection of the previous segment of 'va' over the segment 'v'-'vb' (see the figure). Deletes 'v'. Move 'va' to 'vx' and put this movement in a dictionary 'va'->'vx'. If there are remaining movements iterates over each geometry. If there exist a vertex equal to 'va' move it to 'vx'. Implementation: layer.PolygonLayer.delete_invalid_geometries Algorithm to add topological nodes and simplify duplicate nodes Motivation: Duplicated nodes and Topological errors. Preconditions: This problem occurs both in buildings, building parts, and swimming pools. Its resolution is interdependent, that is, there may be a node of a building that is not duplicated at first but it is if we consider the other data sets. For this reason, the elements of the three data sets must be copied in the same data warehouse. From now, when we say element we refer to buildings, their parts and other constructions (pools). The coordinates are maintained in the original UTM projection to simplify the distance calculation. Parameters: 'dup_thr' = 1.2 cm. This value was established with test taking into account that JOSM draws coordinates with arbitrary precision, but the validation of duplicated nodes round the coordinates to the sevent decimal. An angle of 10-7 degrees is about 0.011 meters (1.1 cm) in latitude. 'dist_thr' = 2 cm. 'straight_thr' = 2°. Steps: For each geometry 'geom' in this layer. For each vertex 'point' in the outermost ring. If this coordinates was not visited previously. Search for geometries intersecting a bounding box of 'dist_thr' radius around 'point'. For each candidate geometry. Search the nearest vertex to 'point' in the candidate geometry. If the distance to the nearest vertex ('dist_v') is 0, it belongs to 'geom'. If any of the adjacent vertices 'va' and 'vb' is near than 'dup_thr' deletes it. If 0 < 'dist_v' < 'dup_thr', it don't belong to 'geom'. Move it to 'point' position. If 'dist_v' > 'dup_thr', it don't belong to 'geom'. 'point' is a topological point if it fullfit this conditions: The distance from 'point' to the nearest segment in the candidate geometry is below 'dist_thr' and the nearest point to 'point' in this segment is not any of their extremes. The angle formed by 'point' with the extremes of the segment is straight (The difference with 180º is below 'ang_thr'). Inserting it as a vertex in the candidate geometry does not generate an invalid geometry. If 'point' is a topological point insert it as a new vertex in the candidate geometry. Topological error not corrected. Topological error corrected. Implementation: layer.PolygonLayer.topology Algorith to simplify geometries Motivation: Unneded nodes. The use of an external tool to simplify does not allow all the control we would like or refine the results. That is why an own algorithm is developed. We intend to eliminate excessive vertices in straight lines, such as the red vertices in the image. Excessive nodes in a straight line. To do this, each vertex of a geometry is examined. The vertex is a candidate to be eliminated if it is not a corner. It is considered that it is not a corner if the angle it forms with the previous and next vertices does not differ much from 180° (it is straight). The red node is a candidate to be deleted. The straight angle condition is not enought. If the distance separating the previous and next vertices is enought large, although the angle is almost straight, the cathetus can measure meters and eliminating the vertex would mean an excessive modification of the building. The cathetus is the shortest distance between the vertex and the line joining the previous and next vertices. Removing this node causes a displacement of several meters. You can not delete a vertex if in another geometry there is a vertex in the same position. That would lead to topological errors. In the following image the green nodes can be eliminated, the red one does not because they belongs to the parts of the building (light gray). Node belonging to several geometries. We have to check the angle in all the geometries that have a node in that position. In the previous image the green nodes belong to two geometries: the footprint of the building and it parts. Preconditions: The elements of the building, parts and other constructions (pools) dataset must be in the same data warehouse. Duplicate nodes have been removed. Topological nodes have been added. The coordinates are maintained in the original UTM projection to simplify the distance calculation. Parameters: 'cath_thr' = 2 cm. Based on the answer on this FAQ: How to express a curved line as a sequence of coordinates? (es) 'straight_thr' = 2°. 'acute_thr' = 10° Steps: Para cada nodo 'point' en la capa. Comprueba si 'point' es una esquina en alguna de las geometrías que tengan un nodo en las mismas coordenadas. Si no es esquina en ninguna, puede borrar 'point' de todas esas geometrías. Implementation: layer.PolygonLayer.simplify Operations on data Algorithm to generate definition files for the task manager Motivation: La creación de proyectos en el gestor de tareas para split of data in tasks. Steps: Se crea un conjunto de datos para rústica 'rustic_zoning' y otro para urbana 'urban_zoning'. Se copian los elementos con valor 'POLIGONO' en el campo levelName del conjunto 'zoning' al conjunto 'rustic_zoning' y los que tienen el valor 'MANZANA' a 'urban_zoning', separando las geometrías multiparte. Se fusionan los elementos 'MANZANA' adyacentes (si tienen algún segmento en común) para evitar colocar en distintas tareas edificios que tengan paredes comunes. Se asigna un identificador único a cada polígono. Results: Se generan dos ficheros de salida: urban_zoning.geojson y rustic_zoning.geojson. Implementation: catatom2osm.CatAtom2Osm.get_zoning Algorithm to split the data into tasks Motivation: Generar ficheros para split of data in tasks. Steps: Para cada característica de tipo edificio o piscina del conjunto de datos 'building'. Si no tiene asignada etiqueta de tarea. Busca una parcela de urbana que lo contenga y le asigna su etiqueta. Si no encuentra, busca un polígono de rústica que lo contenga y le asigna su etiqueta. Las características tipo parte de edificio reciben la misma etiqueta de tarea que su edificio asociado. Una vez asignadas las etiquetas de tarea, se extraen las características que corresponden a cada una y se crean los ficheros OSM correspondientes. Implementation: layer.ConsLayer.set_tasks y catatom2osm.CatAtom2Osm.process_tasks Througfare names conflation Motivation: Corrección de los nombres de viales. Steps: La corrección se realiza en dos fases. En la primera ejecución del programa sobre un municipio, de forma automática: Descargar de OSM las vías y relaciones con las etiquetas highway=* y name=* y las plazas place=square con name=*. Para cada nombre de vial de Catastro. Encontrar en OSM el nombre de vía más similar que esté cerca de los nodos que representan las direcciones con ese nombre de vía. Si no encuentra ninguno, convertir según las reglas de normalización. Las abreviaturas tipo de vía de Catastro (especificadas en este documento) se expanden usando un diccionario, personalizable por el usuario. Generar un archivo de corrección con los nombres de vías en Catastro y la transformación propuesta para cada uno. A continuación se realiza una revisión manual del archivo de corrección. Este archivo será usado por el programa en la siguiente ejecución para transformar los nombres de Catastro. Para algunos tipos de vías, se colocará el nombre en una etiqueta addr:place=* en lugar de addr:street=* Implementación: catatom2osm.CatAtom2Osm.get_translations