OpenMapTiles 3.15.0 SNAPSHOT (#126)

* version bumped from 3.14.0 to 3.15.0-SNAPSHOT

* regenerate-openmaptiles.sh 07f243c5d9efa558fa539d7a31b2ae50507aaa9d (to match content of OMT PR 1457)

* SQL -> Java re-implementation of OMT PR 1457

* version bumped from 3.14.0 to 3.15.0-SNAPSHOT

* WaterName.areaToMinZoom(): improved handling of rounding and precission + added unit tests

* mvn spotless:apply

* water label min. zoom calculation simplified

* comment adjusted to be hopefully more useful

* mvn spotless:apply

* minzoom for CA_TRANSCANADA and US_INTERSTATE trunk now 4 (to match OMT PR 1440)

* minzoom for some other Canada trunks now 4 (to match OMT PR 1446)

* equals() simplified + clean-up of comments

* regenerate-openmaptiles.sh 5f7b2c11b3224759a21133381ca7d959a1f3cf51 (to match content of OMT PR 1465)

* GB road relations processing adjusted to match OMT PR 1465, e.g. handle also primary and secondary roads

* regenerate-openmaptiles.sh edb42f2db3c2b0ec37045367720eed84d7bbd71f (to match content of OMT PR 1466)

* IE road relations processing adjusted to match OMT PR 1466, e.g. handle IE roates in similar way as GB routes

* fixed handling of networkType for secondary GB routes

* clean-up: case statements simplified

* mvn spotless:apply

* clazz calculation moved up so that minzoom can be set to 3 for only lakes (to match OMT PR 1475)

* unit tests adjusted + extended to cover 'minzoom=3 fore lakes' change

* fixed minor typo from previous PR

* render POIs for large universities at low zoom (to match OMT PR 1479)

* clean-up, to make the diff/PR smaller

* regenerate-openmaptiles.sh 5e9b7c475d53a5bd5ea394da361594d3f4ce2d66 (to match content of OMT PR 1485)

* handle 'grade1' and 'tracktype' as per OMT PR 1485

* added implementation of agg_stop

It is based on OMT PR 1480 (which contains latest the fix) and the rest of older code
(which was not worling properly until the fix).

* clean-up: mvn spotless:apply

* Long ferries (as per OMT PR 1486)

* regenerate-openmaptiles.sh b3d67ed5b327c9059aeea0b3304772c6b4c8c7e9  (to match content of OMT PR 1489)

* Add aboriginal lands (as per OMT PR 1489)

* handle duplicate route relations (to match OMT PR 1501)

* regenerate-openmaptiles.sh master, to match several OMT PRs which adjusted only YML

* URLs in comments adjusted to match OMT PR 1560

* Convert separated addresses to dashed addresses

* add brunnel (and layer) attributes only for certain zoomlevels, depending on feature size (matching OMT PR 1579)

* unit test testInterstateMotorway(): brunnel tag for test line no longer available at Z8

* unit test testInterstateMotorway() clean-up: Z13 was tested twice

* minor clean-up: fixed unit test naming

* partial fix for differences in transportation_name layer

The difference is between OpenMapTiles/master (OMT) and
planetiler-openmaptiles/omt_3_15_0 (PT-OMT) (e.g. development versions).

The point is, that while PT-OMT was using limit of "8km" for Z9-Z11, OMT
is using limit "ST_Length(geometry) > 8000 / POWER(2, zoom_level - 9)
AND zoom_level BETWEEN 9 AND 11".

Some further differences still visible, hence further commits expected.

* further adjustments to better match what is done with ferries in OMT

... (as per OMT PR 1486)

But FERRY_MIN_PIXEL_SIZE is "too much" in the contexct of Planetiler,
since it is applied within tiles, hence causes gaps in lines if a line
"strikes a little" certain tile. Hence we will need to divert a little.

* ferry minLength tweak + clean-up

* mvn spotless:apply

* fixed minor typo

* minor reformatting

* ferry line length filter replaced with min. zoom calculation

hence the results are much closer to what OMT is doing for Z4-Z9

* testFerry() adjusted to match previous commit

ferry test polygon with area 1 now qualifies for min. zoom 5

* clea-up of unused stuff + mvn spotless:apply

* mvn spotless:apply

* added TODO node for follow-up pull-request/simplification

* clean-up: common getMinZoom() code moved to Utils

* minzoom clipping for brunnel was adjusted do Z9-Z12 -> test adjusted too

* clean-up

* use same tolerance for all transportation items, like OSM does

* clean-up, since ferry and non-ferry procesing is now same

* we need regenerate to work with master branch for now

* first sub-class search for agg_stop simplified a little

* contains() used instead of indexOf() for better readability

* numbers as list, not array, so that getFirst() and getLast() can be used

* better trimming and filtring of housenumbers

* adjusted handling of large house numbers

* several unit tests collapsed to one with @ParameterizedTest + @CsvSource

* AGG_STOP_SUBCLASS_ORDER simplified from Map to List

* fixed major omission from previous commit

* clamp() used to replace min()&max() combo

* agg_stop now implemented

* fixed typo in the error message

* prepare IE and GB boundary geometry outside of synchronized{}

* fixed typo in the error message

* mvn spotless:apply

* switch statements for IE and GB route networks simplified

* avoid RouteNetwork->String mapping, not needed for anyMatch()

* fix: attr. brunnel optional based on size on Z4-Z11, attr. layer optional between Z9-Z11

* tolerance change in transportation reverted, added note to README as per why

* fix: monzoom for sea&co. is Z0-Z14 based on area, for the rest it is Z3-Z14 again based on area

* clean-up: avoid doing area->side->area, do just area

* regenerate-openmaptiles.sh 6c31841f4674f15e15afde346a060cf7c22e6cdd (to match content of OMT PR 1591)

* relevant process() functions adjusted to match changes in transportation/mapping.yaml

* regenerate-openmaptiles.sh master, instead of 6c31841f4674f15e15afde346a060cf7c22e6cdd (to match content of OMT PR 1591, in a cleaner way)

* introduce duplicate housenumber filtering (matching OMT PR 1391)

* (less related) clean-up: use isEmpty() instead if size check

* testContainsHousenumber UT adjusted, since duplicate housenumber filtering is reducing amount of house numbers

* use combination of uic_ref, name, network and operator as key for agg_stop sets

If we rely on only on `uic_ref` we group together also stations which are
too far apart (even different cities). With this combo results seem OK,
e.g. all grouped stations are within around 950m (1000 pixels at Z14) of
each other (1000 being used in `PARTITION BY LabelGrid(...` in
`layers/poi/poi.sql` in OpenMapTiles).

* agg_stop comparison made more explicit, since we want to match same exact one

* mvn spotless:apply

* name now important for agg_stop processing, hence name:es (ab)used for unit tests

* agg_stop: simplified processing of nearest station

Results still same, only ordering is different:
- previously: agg_stop=1 first
- now: FIFO

* agg_stop: forther code simplification

* fixed major typo introduced in previous merge

* setMinPixelSize() + setMinZoom() used instead of areaToMinZoom()

* clean-up: unused stuff removed

* mvn spotless:apply

* setAttrWithMinSize() used instead of getBrunnelMinzoom()

getFerryMinzoom() kept since we'd like to replicate `sql_filter: ST_Length(...` from OMT

* getMinZoomForLength() no longer used, hence removed

* clean-up: LOG2 not used, hence removed

* added BY_TEMP_HAS_NAME comparator to avoid its repeated construction during run-time

* duplicate houcenumber processing simplified further

* clean-up: get(0) replaced with getFirst()

* clean-up: CPU-intensive prepare() moved out of synchronized block

* regenerate-openmaptiles.sh 3cf77e2a542d8a369bb08bf2538cdde0b3effb2b (to match content of OMT PR 1423)

* unit test adjusted for POI office class changes

* regenerate-openmaptiles.sh master (to match content of OMT PR 1544)

* added charging_station implementation matching OMT PR 1544

* use setMinPixelSizeBelowZoom() instead of uniAreaToMinZoom()

* use setMinPixelSizeBelowZoom() instead of getFerryMinzoom()

* fixed unit test, to match recent tweaks
This commit is contained in:
Peter Hanecak
2023-12-22 10:23:37 +01:00
committed by GitHub
parent 58e2c89500
commit add205e26c
21 changed files with 1719 additions and 259 deletions

View File

@@ -43,9 +43,18 @@ import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.geo.GeometryException;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.Translations;
import java.util.Arrays;
import java.util.Comparator;
import java.util.List;
import java.util.function.Predicate;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
import org.openmaptiles.generated.OpenMapTilesSchema;
import org.openmaptiles.generated.Tables;
import org.openmaptiles.util.Utils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* Defines the logic for generating map elements in the {@code housenumber} layer from source features.
@@ -59,19 +68,96 @@ public class Housenumber implements
Tables.OsmHousenumberPoint.Handler,
ForwardingProfile.FeaturePostProcessor {
public Housenumber(Translations translations, PlanetilerConfig config, Stats stats) {}
private static final Logger LOGGER = LoggerFactory.getLogger(Housenumber.class);
private static final String OSM_SEPARATOR = ";";
private static final String DISPLAY_SEPARATOR = "";
private static final Pattern NO_CONVERSION_PATTERN = Pattern.compile("[^0-9;]");
private static final String TEMP_PARTITION = "_partition";
private static final String TEMP_HAS_NAME = "_has_name";
private static final Comparator<VectorTile.Feature> BY_TEMP_HAS_NAME = Comparator
.comparing(i -> (Boolean) i.attrs().get(TEMP_HAS_NAME), Boolean::compare);
private final Stats stats;
public Housenumber(Translations translations, PlanetilerConfig config, Stats stats) {
this.stats = stats;
}
private static String displayHousenumberNonumeric(List<String> numbers) {
return numbers.getFirst()
.concat(DISPLAY_SEPARATOR)
.concat(numbers.getLast());
}
protected static String displayHousenumber(String housenumber) {
if (!housenumber.contains(OSM_SEPARATOR)) {
return housenumber;
}
List<String> numbers = Arrays.stream(housenumber.split(OSM_SEPARATOR))
.map(String::trim)
.filter(Predicate.not(String::isEmpty))
.toList();
if (numbers.isEmpty()) {
// not much to do with strange/invalid entries like "3;" or ";" etc.
return housenumber;
}
Matcher matcher = NO_CONVERSION_PATTERN.matcher(housenumber);
if (matcher.find()) {
return displayHousenumberNonumeric(numbers);
}
// numeric display house number
var statistics = numbers.stream()
.collect(Collectors.summarizingLong(Long::parseUnsignedLong));
return String.valueOf(statistics.getMin())
.concat(DISPLAY_SEPARATOR)
.concat(String.valueOf(statistics.getMax()));
}
@Override
public void process(Tables.OsmHousenumberPoint element, FeatureCollector features) {
String housenumber;
try {
housenumber = displayHousenumber(element.housenumber());
} catch (NumberFormatException e) {
// should not be happening (thanks to NO_CONVERSION_PATTERN) but ...
stats.dataError("housenumber_range");
LOGGER.warn("Failed to convert housenumber range: {}", element.housenumber());
housenumber = element.housenumber();
}
String partition = Utils.coalesce(element.street(), "")
.concat(Utils.coalesce(element.blockNumber(), ""))
.concat(housenumber);
Boolean hasName = element.hasName() == null ? Boolean.FALSE : !element.hasName().isEmpty();
features.centroidIfConvex(LAYER_NAME)
.setBufferPixels(BUFFER_SIZE)
.setAttr(Fields.HOUSENUMBER, element.housenumber())
.setAttr(Fields.HOUSENUMBER, housenumber)
.setAttr(TEMP_PARTITION, partition)
.setAttr(TEMP_HAS_NAME, hasName)
.setMinZoom(14);
}
@Override
public List<VectorTile.Feature> postProcess(int zoom, List<VectorTile.Feature> list) throws GeometryException {
// remove duplicate house numbers, features without name tag are prioritized
var items = list.stream()
.collect(Collectors.groupingBy(f -> f.attrs().get(TEMP_PARTITION)))
.values().stream()
.flatMap(
g -> g.stream().min(BY_TEMP_HAS_NAME).stream()
)
.toList();
// remove temporary attributes
for (var item : items) {
item.attrs().remove(TEMP_HAS_NAME);
item.attrs().remove(TEMP_PARTITION);
}
// reduces the size of some heavy z14 tiles with many repeated housenumber values by 60% or more
return FeatureMerge.mergeMultiPoint(list);
return FeatureMerge.mergeMultiPoint(items);
}
}

View File

@@ -91,15 +91,20 @@ public class Park implements
@Override
public void process(Tables.OsmParkPolygon element, FeatureCollector features) {
String protectionTitle = element.protectionTitle();
if (protectionTitle != null) {
protectionTitle = protectionTitle.replace(' ', '_').toLowerCase(Locale.ROOT);
String clazz;
if ("aboriginal_lands".equals(element.boundary())) {
clazz = "aboriginal_lands";
} else {
String protectionTitle = element.protectionTitle();
if (protectionTitle != null) {
protectionTitle = protectionTitle.replace(' ', '_').toLowerCase(Locale.ROOT);
}
clazz = coalesce(
nullIfEmpty(protectionTitle),
nullIfEmpty(element.boundary()),
nullIfEmpty(element.leisure())
);
}
String clazz = coalesce(
nullIfEmpty(protectionTitle),
nullIfEmpty(element.boundary()),
nullIfEmpty(element.leisure())
);
// park shape
var outline = features.polygon(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
@@ -138,7 +143,7 @@ public class Park implements
// sql filter: area > 70000*2^(20-zoom_level)
// simplifies to: zoom_level > 20 - log(area / 70000) / log(2)
int minzoom = (int) Math.floor(20 - Math.log(area / WORLD_AREA_FOR_70K_SQUARE_METERS) / LOG2);
minzoom = Math.min(14, Math.max(5, minzoom));
minzoom = Math.clamp(minzoom, 5, 14);
return minzoom;
}

View File

@@ -235,7 +235,7 @@ public class Place implements
rank = country.rank;
}
rank = Math.max(1, Math.min(6, rank));
rank = Math.clamp(rank, 1, 6);
features.point(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.putAttrs(names)
@@ -261,7 +261,7 @@ public class Place implements
if (nullOrEmpty(names.get(Fields.NAME_EN))) {
names.put(Fields.NAME_EN, state.name);
}
int rank = Math.min(6, Math.max(1, state.rank));
int rank = Math.clamp(state.rank, 1, 6);
features.point(LAYER_NAME).setBufferPixels(BUFFER_SIZE)
.putAttrs(names)

View File

@@ -48,14 +48,26 @@ import com.onthegomap.planetiler.VectorTile;
import com.onthegomap.planetiler.collection.Hppc;
import com.onthegomap.planetiler.config.PlanetilerConfig;
import com.onthegomap.planetiler.expression.MultiExpression;
import com.onthegomap.planetiler.geo.GeoUtils;
import com.onthegomap.planetiler.geo.GeometryException;
import com.onthegomap.planetiler.reader.SimpleFeature;
import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.Parse;
import com.onthegomap.planetiler.util.Translations;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.function.Consumer;
import org.locationtech.jts.geom.Point;
import org.openmaptiles.OpenMapTilesProfile;
import org.openmaptiles.generated.OpenMapTilesSchema;
import org.openmaptiles.generated.Tables;
import org.openmaptiles.util.OmtLanguageUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* Defines the logic for generating map elements for things like shops, parks, and schools in the {@code poi} layer from
@@ -68,13 +80,15 @@ public class Poi implements
OpenMapTilesSchema.Poi,
Tables.OsmPoiPoint.Handler,
Tables.OsmPoiPolygon.Handler,
ForwardingProfile.FeaturePostProcessor {
ForwardingProfile.FeaturePostProcessor,
OpenMapTilesProfile.FinishHandler {
/*
* process() creates the raw POI feature from OSM elements and postProcess()
* assigns the feature rank from order in the tile at render-time.
*/
private static final Logger LOGGER = LoggerFactory.getLogger(Poi.class);
private static final Map<String, Integer> CLASS_RANKS = Map.ofEntries(
entry(FieldValues.CLASS_HOSPITAL, 20),
entry(FieldValues.CLASS_RAILWAY, 40),
@@ -99,12 +113,25 @@ public class Poi implements
entry(FieldValues.CLASS_CLOTHING_STORE, 700),
entry(FieldValues.CLASS_BAR, 800)
);
private static final Set<String> UNIVERSITY_POI_SUBCLASSES = Set.of("university", "college");
private static final List<String> AGG_STOP_SUBCLASS_ORDER = List.of(
"subway",
"tram_stop",
"bus_station",
"bus_stop"
);
private static final Comparator<Tables.OsmPoiPoint> BY_SUBCLASS = Comparator
.comparingInt(s -> AGG_STOP_SUBCLASS_ORDER.indexOf(s.subclass()));
private static final Set<String> BRAND_OPERATOR_REF_SUBCLASSES = Set.of("charging_station", "parcel_locker");
private final MultiExpression.Index<String> classMapping;
private final Translations translations;
private final Stats stats;
private final Map<String, List<Tables.OsmPoiPoint>> aggStops = new HashMap<>();
public Poi(Translations translations, PlanetilerConfig config, Stats stats) {
this.classMapping = FieldMappings.Class.index();
this.translations = translations;
this.stats = stats;
}
static int poiClassRank(String clazz) {
@@ -125,19 +152,112 @@ public class Poi implements
return lowZoom ? 12 : 14;
}
@Override
public void release() {
aggStops.clear();
}
@Override
public void process(Tables.OsmPoiPoint element, FeatureCollector features) {
// TODO handle uic_ref => agg_stop
setupPoiFeature(element, features.point(LAYER_NAME));
if (element.uicRef() != null && AGG_STOP_SUBCLASS_ORDER.contains(element.subclass())) {
// multiple threads may update this concurrently
String aggStopKey = element.uicRef()
.concat(coalesce(nullIfEmpty(element.name()), ""))
.concat(coalesce(nullIfEmpty(element.network()), ""))
.concat(coalesce(nullIfEmpty(element.operator()), ""));
synchronized (this) {
aggStops.computeIfAbsent(aggStopKey, key -> new ArrayList<>()).add(element);
}
} else {
setupPoiFeature(element, features.point(LAYER_NAME), null);
}
}
private void processAggStop(Tables.OsmPoiPoint element, FeatureCollector.Factory featureCollectors,
Consumer<FeatureCollector.Feature> emit, Integer aggStop) {
try {
var features = featureCollectors.get(SimpleFeature.fromWorldGeometry(element.source().worldGeometry()));
setupPoiFeature(element, features.point(LAYER_NAME), aggStop);
for (var feature : features) {
emit.accept(feature);
}
} catch (GeometryException e) {
e.log(stats, "agg_stop_geometry_2",
"Error getting geometry for the stop " + element.source().id() + " (agg_stop)");
}
}
/**
* We've put aside some stops for {@code agg_stop} processing and we do that processing here.
* <p>
* The main point is to group together stops with same {@code uid_ref} and then order them first based on subclass
* (see {@code AGG_STOP_ORDER}) and then based on distance from centroid (calculated from all the stops). The first
* one gets {@code agg_stop=1}, the rest will be "normal" (e.g. no {@code agg_stop} attribute).
* <p>
* ref: <a href=
* "https://github.com/openmaptiles/openmaptiles/blob/master/layers/poi/poi_stop_agg.sql#L26,L28">poi_stop_agg.sql</a>
*/
@Override
public void finish(String sourceName, FeatureCollector.Factory featureCollectors,
Consumer<FeatureCollector.Feature> emit) {
if (OpenMapTilesProfile.OSM_SOURCE.equals(sourceName)) {
var timer = stats.startStage("agg_stop");
LOGGER.info("Processing {} agg_stop sets", aggStops.size());
for (var aggStopSet : aggStops.values()) {
if (aggStopSet.size() == 1) {
processAggStop(aggStopSet.getFirst(), featureCollectors, emit, 1);
continue;
}
Tables.OsmPoiPoint nearest = null;
try {
// find most important stops based on subclass
var firstSubclass = aggStopSet.stream().min(BY_SUBCLASS).get().subclass();
var topAggStops =
aggStopSet.stream().filter(s -> firstSubclass.equals(s.subclass())).toArray(Tables.OsmPoiPoint[]::new);
// calculate the centroid and ...
List<Point> aggStopPoints = new ArrayList<>(aggStopSet.size());
for (var aggStop : aggStopSet) {
aggStopPoints.add(aggStop.source().worldGeometry().getCentroid());
}
var aggStopCentroid = GeoUtils.combinePoints(aggStopPoints).getCentroid();
// ... find one stop nearest to the centroid
double minDistance = Double.MAX_VALUE;
for (var aggStop : topAggStops) {
double distance = aggStopCentroid.distance(aggStop.source().worldGeometry());
if (distance < minDistance) {
minDistance = distance;
nearest = aggStop;
}
}
} catch (GeometryException e) {
e.log(stats, "agg_stop_geometry_1",
"Error getting geometry for some of the stops with UIC ref. " + aggStopSet.getFirst().uicRef() +
" (agg_stop)");
// we're not able to calculate agg_stop, so simply dump the stops as they are
nearest = null;
}
// now emit the stops
final Tables.OsmPoiPoint nearestFinal = nearest; // final needed for lambda
aggStopSet
.forEach(s -> processAggStop(s, featureCollectors, emit, s == nearestFinal ? 1 : null));
}
timer.stop();
}
}
@Override
public void process(Tables.OsmPoiPolygon element, FeatureCollector features) {
setupPoiFeature(element, features.centroidIfConvex(LAYER_NAME));
setupPoiFeature(element, features.centroidIfConvex(LAYER_NAME), null);
}
private <T extends Tables.WithSubclass & Tables.WithStation & Tables.WithFunicular & Tables.WithSport & Tables.WithInformation & Tables.WithReligion & Tables.WithMappingKey & Tables.WithName & Tables.WithIndoor & Tables.WithLayer & Tables.WithSource & Tables.WithOperator & Tables.WithNetwork & Tables.WithBrand & Tables.WithRef> void setupPoiFeature(
T element, FeatureCollector.Feature output) {
T element, FeatureCollector.Feature output, Integer aggStop) {
String rawSubclass = element.subclass();
if ("station".equals(rawSubclass) && "subway".equals(element.station())) {
rawSubclass = "subway";
@@ -157,7 +277,7 @@ public class Poi implements
}
// Parcel locker without name: use either brand or operator and add ref if present
if ("parcel_locker".equals(rawSubclass) && nullOrEmpty(name)) {
if (BRAND_OPERATOR_REF_SUBCLASSES.contains(rawSubclass) && nullOrEmpty(name)) {
name = coalesce(nullIfEmpty(element.brand()), nullIfEmpty(element.operator()));
String ref = nullIfEmpty(element.ref());
if (ref != null) {
@@ -178,16 +298,24 @@ public class Poi implements
int poiClassRank = poiClassRank(poiClass);
int rankOrder = poiClassRank + ((nullOrEmpty(name)) ? 2000 : 0);
int minzoom = minzoom(element.subclass(), element.mappingKey());
if (UNIVERSITY_POI_SUBCLASSES.contains(rawSubclass)) {
// universities that are at least 10% of a tile may appear from Z10
output.setMinPixelSizeBelowZoom(13, 80); // 80x80px is ~10% of a 256x256px tile
minzoom = 10;
}
output.setBufferPixels(BUFFER_SIZE)
.setAttr(Fields.CLASS, poiClass)
.setAttr(Fields.SUBCLASS, subclass)
.setAttr(Fields.LAYER, nullIfLong(element.layer(), 0))
.setAttr(Fields.LEVEL, Parse.parseLongOrNull(element.source().getTag("level")))
.setAttr(Fields.INDOOR, element.indoor() ? 1 : null)
.setAttr(Fields.AGG_STOP, aggStop)
.putAttrs(OmtLanguageUtils.getNames(element.source().tags(), translations))
.setPointLabelGridPixelSize(14, 64)
.setSortKey(rankOrder)
.setMinZoom(minzoom(element.subclass(), element.mappingKey()));
.setMinZoom(minzoom);
}
@Override

View File

@@ -62,6 +62,7 @@ import java.util.Collections;
import java.util.Comparator;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.regex.Matcher;
@@ -103,7 +104,8 @@ public class Transportation implements
*/
private static final Logger LOGGER = LoggerFactory.getLogger(Transportation.class);
private static final Pattern GREAT_BRITAIN_REF_NETWORK_PATTERN = Pattern.compile("^[AM][0-9AM()]+");
private static final Pattern GREAT_BRITAIN_REF_NETWORK_PATTERN = Pattern.compile("^[ABM][0-9ABM()]+");
private static final Pattern IRELAND_REF_NETWORK_PATTERN = Pattern.compile("^[MNRL][0-9]+");
private static final MultiExpression.Index<String> classMapping = FieldMappings.Class.index();
private static final Set<String> RAILWAY_RAIL_VALUES = Set.of(
FieldValues.SUBCLASS_RAIL,
@@ -132,11 +134,22 @@ public class Transportation implements
);
private static final Set<String> SURFACE_PAVED_VALUES = Set.of(
"paved", "asphalt", "cobblestone", "concrete", "concrete:lanes", "concrete:plates", "metal",
"paving_stones", "sett", "unhewn_cobblestone", "wood"
"paving_stones", "sett", "unhewn_cobblestone", "wood", "grade1"
);
private static final Set<String> ACCESS_NO_VALUES = Set.of(
"private", "no"
);
private static final Set<RouteNetwork> TRUNK_AS_MOTORWAY_BY_NETWORK = Set.of(
RouteNetwork.CA_TRANSCANADA,
RouteNetwork.CA_PROVINCIAL_ARTERIAL,
RouteNetwork.US_INTERSTATE
);
private static final Set<String> CA_AB_PRIMARY_AS_ARTERIAL_BY_REF = Set.of(
"2", "3", "4"
);
private static final Set<String> CA_BC_AS_ARTERIAL_BY_REF = Set.of(
"3", "5", "99"
);
private static final ZoomFunction.MeterToPixelThresholds MIN_LENGTH = ZoomFunction.meterThresholds()
.put(7, 50)
.put(6, 100)
@@ -150,13 +163,15 @@ public class Transportation implements
.thenComparingInt(r -> r.ref().length())
.thenComparing(RouteRelation::ref);
private static final Set<Integer> ONEWAY_VALUES = Set.of(-1, 1);
private final Map<String, Integer> MINZOOMS;
private static final String LIMIT_MERGE_TAG = "__limit_merge";
private final AtomicBoolean loggedNoGb = new AtomicBoolean(false);
private final AtomicBoolean loggedNoIreland = new AtomicBoolean(false);
private final boolean z13Paths;
private final Map<String, Integer> MINZOOMS;
private final Stats stats;
private final PlanetilerConfig config;
private PreparedGeometry greatBritain = null;
private PreparedGeometry ireland = null;
public Transportation(Translations translations, PlanetilerConfig config, Stats stats) {
this.config = config;
@@ -239,16 +254,32 @@ public class Transportation implements
@Override
public void processNaturalEarth(String table, SourceFeature feature,
FeatureCollector features) {
if ("ne_10m_admin_0_countries".equals(table) && feature.hasTag("iso_a2", "GB")) {
// multiple threads call this method concurrently, GB polygon *should* only be found
// once, but just to be safe synchronize updates to that field
synchronized (this) {
try {
Geometry boundary = feature.polygon().buffer(GeoUtils.metersToPixelAtEquator(0, 10_000) / 256d);
greatBritain = PreparedGeometryFactory.prepare(boundary);
} catch (GeometryException e) {
LOGGER.error("Failed to get Great Britain Polygon: " + e);
if (!"ne_10m_admin_0_countries".equals(table)) {
return;
}
// multiple threads call this method concurrently, GB (or IE) polygon *should* only be found
// once, but just to be safe synchronize updates to that field
if (feature.hasTag("iso_a2", "GB")) {
try {
var prepared = PreparedGeometryFactory.prepare(
feature.polygon().buffer(GeoUtils.metersToPixelAtEquator(0, 10_000) / 256d)
);
synchronized (this) {
greatBritain = prepared;
}
} catch (GeometryException e) {
LOGGER.error("Failed to get Great Britain Polygon: " + e);
}
} else if (feature.hasTag("iso_a2", "IE")) {
try {
var prepared = PreparedGeometryFactory.prepare(
feature.polygon().buffer(GeoUtils.metersToPixelAtEquator(0, 10_000) / 256d)
);
synchronized (this) {
ireland = prepared;
}
} catch (GeometryException e) {
LOGGER.error("Failed to get Ireland Polygon: " + e);
}
}
}
@@ -268,6 +299,26 @@ public class Transportation implements
networkType = RouteNetwork.US_STATE;
} else if (network != null && network.startsWith("CA:transcanada")) {
networkType = RouteNetwork.CA_TRANSCANADA;
} else if ("CA:QC:A".equals(network)) {
networkType = RouteNetwork.CA_PROVINCIAL_ARTERIAL;
} else if ("CA:ON:primary".equals(network)) {
if (ref != null && ref.length() == 3 && ref.startsWith("4")) {
networkType = RouteNetwork.CA_PROVINCIAL_ARTERIAL;
} else if ("QEW".equals(ref)) {
networkType = RouteNetwork.CA_PROVINCIAL_ARTERIAL;
} else {
networkType = RouteNetwork.CA_PROVINCIAL;
}
} else if ("CA:MB:PTH".equals(network) && "75".equals(ref)) {
networkType = RouteNetwork.CA_PROVINCIAL_ARTERIAL;
} else if ("CA:AB:primary".equals(network) && ref != null && CA_AB_PRIMARY_AS_ARTERIAL_BY_REF.contains(ref)) {
networkType = RouteNetwork.CA_PROVINCIAL_ARTERIAL;
} else if ("CA:BC".equals(network) && ref != null && CA_BC_AS_ARTERIAL_BY_REF.contains(ref)) {
networkType = RouteNetwork.CA_PROVINCIAL_ARTERIAL;
} else if (network != null && ((network.length() == 5 && network.startsWith("CA:")) ||
(network.length() >= 6 && network.startsWith("CA:") && network.charAt(5) == ':'))) {
// in SQL: LIKE 'CA:__' OR network LIKE 'CA:__:%'; but wanted to avoid regexp hence more ugly
networkType = RouteNetwork.CA_PROVINCIAL;
}
int rank = switch (coalesce(network, "")) {
@@ -307,12 +358,15 @@ public class Transportation implements
try {
Geometry wayGeometry = element.source().worldGeometry();
if (greatBritain.intersects(wayGeometry)) {
Transportation.RouteNetwork networkType =
"motorway".equals(element.highway()) ? Transportation.RouteNetwork.GB_MOTORWAY :
Transportation.RouteNetwork.GB_TRUNK;
String network = "motorway".equals(element.highway()) ? "omt-gb-motorway" : "omt-gb-trunk";
result.add(new RouteRelation(refMatcher.group(), network, networkType, (byte) -1,
0));
Transportation.RouteNetwork networkType = switch (element.highway()) {
case "motorway" -> Transportation.RouteNetwork.GB_MOTORWAY;
case "trunk" -> RouteNetwork.GB_TRUNK;
case "primary", "secondary" -> RouteNetwork.GB_PRIMARY;
default -> null;
};
result.add(new RouteRelation(refMatcher.group(),
networkType == null ? null : networkType.network,
networkType, (byte) -1, 0));
}
} catch (GeometryException e) {
e.log(stats, "omt_transportation_name_gb_test",
@@ -320,6 +374,31 @@ public class Transportation implements
}
}
}
// Similarly Ireland.
refMatcher = IRELAND_REF_NETWORK_PATTERN.matcher(ref);
if (refMatcher.find()) {
if (ireland == null) {
if (!loggedNoIreland.get() && loggedNoIreland.compareAndSet(false, true)) {
LOGGER.warn("No IE polygon for inferring route network types");
}
} else {
try {
Geometry wayGeometry = element.source().worldGeometry();
if (ireland.intersects(wayGeometry)) {
String highway = coalesce(element.highway(), "");
Transportation.RouteNetwork networkType = switch (highway) {
case "motorway" -> Transportation.RouteNetwork.IE_MOTORWAY;
case "trunk", "primary" -> RouteNetwork.IE_NATIONAL;
default -> RouteNetwork.IE_REGIONAL;
};
result.add(new RouteRelation(refMatcher.group(), networkType.network, networkType, (byte) -1, 0));
}
} catch (GeometryException e) {
e.log(stats, "omt_transportation_name_ie_test",
"Unable to test highway against IE route network: " + element.source().id());
}
}
}
}
Collections.sort(result);
return result;
@@ -362,13 +441,11 @@ public class Transportation implements
.setAttr(Fields.CLASS, highwayClass)
.setAttr(Fields.SUBCLASS, highwaySubclass(highwayClass, element.publicTransport(), highway))
.setAttr(Fields.NETWORK, networkType != null ? networkType.name : null)
// TODO: including brunnel at low zooms leads to some large 300-400+kb z4-7 tiles, instead
// we should only set brunnel if the line is above a certain length
.setAttr(Fields.BRUNNEL, brunnel(element.isBridge(), element.isTunnel(), element.isFord()))
.setAttrWithMinSize(Fields.BRUNNEL, brunnel(element.isBridge(), element.isTunnel(), element.isFord()), 4, 4, 12)
// z8+
.setAttrWithMinzoom(Fields.EXPRESSWAY, element.expressway() && !"motorway".equals(highway) ? 1 : null, 8)
// z9+
.setAttrWithMinzoom(Fields.LAYER, nullIfLong(element.layer(), 0), 9)
.setAttrWithMinSize(Fields.LAYER, nullIfLong(element.layer(), 0), 4, 9, 12)
.setAttrWithMinzoom(Fields.BICYCLE, nullIfEmpty(element.bicycle()), 9)
.setAttrWithMinzoom(Fields.FOOT, nullIfEmpty(element.foot()), 9)
.setAttrWithMinzoom(Fields.HORSE, nullIfEmpty(element.horse()), 9)
@@ -381,7 +458,7 @@ public class Transportation implements
// z12+
.setAttrWithMinzoom(Fields.SERVICE, service, 12)
.setAttrWithMinzoom(Fields.ONEWAY, nullIfInt(element.isOneway(), 0), 12)
.setAttrWithMinzoom(Fields.SURFACE, surface(element.surface()), 12)
.setAttrWithMinzoom(Fields.SURFACE, surface(coalesce(element.surface(), element.tracktype())), 12)
.setMinPixelSize(0) // merge during post-processing, then limit by size
.setSortKey(element.zOrder())
.setMinZoom(minzoom);
@@ -415,6 +492,14 @@ public class Transportation implements
case FieldValues.CLASS_SERVICE -> isDrivewayOrParkingAisle(service(element.service())) ? 14 : 13;
case FieldValues.CLASS_TRACK, FieldValues.CLASS_PATH -> routeRank == 1 ? 12 :
(z13Paths || !nullOrEmpty(element.name()) || routeRank <= 2 || !nullOrEmpty(element.sacScale())) ? 13 : 14;
case FieldValues.CLASS_TRUNK -> {
// trunks in some networks to have same min. zoom as highway = "motorway"
String clazz = routeRelations.stream()
.map(RouteRelation::networkType)
.filter(Objects::nonNull)
.anyMatch(TRUNK_AS_MOTORWAY_BY_NETWORK::contains) ? FieldValues.CLASS_MOTORWAY : FieldValues.CLASS_TRUNK;
yield MINZOOMS.getOrDefault(clazz, Integer.MAX_VALUE);
}
default -> MINZOOMS.getOrDefault(baseClass, Integer.MAX_VALUE);
};
}
@@ -463,7 +548,6 @@ public class Transportation implements
.setAttr(Fields.CLASS, clazz)
.setAttr(Fields.SUBCLASS, railway)
.setAttr(Fields.SERVICE, service(service))
.setAttr(Fields.ONEWAY, nullIfInt(element.isOneway(), 0))
.setAttr(Fields.RAMP, element.isRamp() ? 1L : null)
.setAttrWithMinzoom(Fields.BRUNNEL, brunnel(element.isBridge(), element.isTunnel(), element.isFord()), 10)
.setAttrWithMinzoom(Fields.LAYER, nullIfLong(element.layer(), 0), 9)
@@ -494,13 +578,13 @@ public class Transportation implements
.setAttr(Fields.CLASS, element.shipway()) // "ferry"
// no subclass
.setAttr(Fields.SERVICE, service(element.service()))
.setAttr(Fields.ONEWAY, nullIfInt(element.isOneway(), 0))
.setAttr(Fields.RAMP, element.isRamp() ? 1L : null)
.setAttr(Fields.BRUNNEL, brunnel(element.isBridge(), element.isTunnel(), element.isFord()))
.setAttr(Fields.LAYER, nullIfLong(element.layer(), 0))
.setSortKey(element.zOrder())
.setMinPixelSize(0) // merge during post-processing, then limit by size
.setMinZoom(11);
.setMinZoom(4)
.setMinPixelSizeBelowZoom(10, 32); // `sql_filter: ST_Length(...)` used in OpenMapTiles translates to 32px
}
@Override
@@ -548,17 +632,25 @@ public class Transportation implements
enum RouteNetwork {
US_INTERSTATE("us-interstate"),
US_HIGHWAY("us-highway"),
US_STATE("us-state"),
CA_TRANSCANADA("ca-transcanada"),
GB_MOTORWAY("gb-motorway"),
GB_TRUNK("gb-trunk");
US_INTERSTATE("us-interstate", null),
US_HIGHWAY("us-highway", null),
US_STATE("us-state", null),
CA_TRANSCANADA("ca-transcanada", null),
CA_PROVINCIAL_ARTERIAL("ca-provincial-arterial", null),
CA_PROVINCIAL("ca-provincial", null),
GB_MOTORWAY("gb-motorway", "omt-gb-motorway"),
GB_TRUNK("gb-trunk", "omt-gb-trunk"),
GB_PRIMARY("gb-primary", "omt-gb-primary"),
IE_MOTORWAY("ie-motorway", "omt-ie-motorway"),
IE_NATIONAL("ie-national", "omt-ie-national"),
IE_REGIONAL("ie-regional", "omt-ie-regional");
final String name;
final String network;
RouteNetwork(String name) {
RouteNetwork(String name, String network) {
this.name = name;
this.network = network;
}
}

View File

@@ -111,8 +111,8 @@ public class TransportationName implements
.put(7, 20_000)
.put(8, 14_000)
.put(9, 8_000)
.put(10, 8_000)
.put(11, 8_000);
.put(10, 4_000)
.put(11, 2_000);
private final boolean brunnel;
private final boolean sizeForShield;
private final boolean limitMerge;
@@ -279,7 +279,9 @@ public class TransportationName implements
}
if (brunnel) {
feature.setAttr(Fields.BRUNNEL, brunnel(element.isBridge(), element.isTunnel(), element.isFord()));
// from OMT: "Drop brunnel if length of way < 2% of tile width (less than 3 pixels)"
feature.setAttrWithMinSize(Fields.BRUNNEL, brunnel(element.isBridge(), element.isTunnel(), element.isFord()),
3, 4, 12);
}
/*

View File

@@ -35,6 +35,7 @@ See https://github.com/openmaptiles/openmaptiles/blob/master/LICENSE.md for deta
*/
package org.openmaptiles.layers;
import static org.openmaptiles.util.Utils.coalesce;
import static org.openmaptiles.util.Utils.nullIfEmpty;
import com.carrotsearch.hppc.LongObjectMap;
@@ -48,6 +49,7 @@ import com.onthegomap.planetiler.stats.Stats;
import com.onthegomap.planetiler.util.Parse;
import com.onthegomap.planetiler.util.Translations;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.ConcurrentSkipListMap;
import org.locationtech.jts.geom.Geometry;
import org.openmaptiles.OpenMapTilesProfile;
@@ -80,9 +82,7 @@ public class WaterName implements
*/
private static final Logger LOGGER = LoggerFactory.getLogger(WaterName.class);
private static final double WORLD_AREA_FOR_70K_SQUARE_METERS =
Math.pow(GeoUtils.metersToPixelAtEquator(0, Math.sqrt(70_000)) / 256d, 2);
private static final double LOG2 = Math.log(2);
private static final Set<String> SEA_OR_OCEAN_PLACE = Set.of("sea", "ocean");
private final Translations translations;
// need to synchronize updates from multiple threads
private final LongObjectMap<Geometry> lakeCenterlines = Hppc.newLongObjectHashMap();
@@ -141,7 +141,10 @@ public class WaterName implements
@Override
public void process(Tables.OsmMarinePoint element, FeatureCollector features) {
if (!element.name().isBlank()) {
String place = element.place();
String clazz = coalesce(
nullIfEmpty(element.natural()),
nullIfEmpty(element.place())
);
var source = element.source();
// use name from OSM, but get min zoom from natural earth based on fuzzy name match...
Integer rank = Parse.parseIntOrNull(source.getTag("rank"));
@@ -159,11 +162,25 @@ public class WaterName implements
rank = next.getValue();
}
}
int minZoom = "ocean".equals(place) ? 0 : rank != null ? rank : 8;
int minZoom;
if ("ocean".equals(element.place())) {
minZoom = 0;
} else if (rank != null) {
// FIXME: While this looks like matching properly stuff in https://github.com/openmaptiles/openmaptiles/pull/1457/files#diff-201daa1c61c99073fe3280d440c9feca5ed2236b251ad454caa14cc203f952d1R74 ,
// it includes not just https://www.openstreetmap.org/relation/13360255 but also https://www.openstreetmap.org/node/1385157299 (and some others).
// Hence check how that OpenMapTiles code works for "James Bay" and:
// a) if same as here then, fix there and then here
// b) if OK (while here NOK), fix only here
minZoom = rank;
} else if ("bay".equals(element.natural())) {
minZoom = 13;
} else {
minZoom = 8;
}
features.point(LAYER_NAME)
.setBufferPixels(BUFFER_SIZE)
.putAttrs(OmtLanguageUtils.getNames(source.tags(), translations))
.setAttr(Fields.CLASS, place)
.setAttr(Fields.CLASS, clazz)
.setAttr(Fields.INTERMITTENT, element.isIntermittent() ? 1 : 0)
.setMinZoom(minZoom);
}
@@ -172,31 +189,35 @@ public class WaterName implements
@Override
public void process(Tables.OsmWaterPolygon element, FeatureCollector features) {
if (nullIfEmpty(element.name()) != null) {
try {
Geometry centerlineGeometry = lakeCenterlines.get(element.source().id());
FeatureCollector.Feature feature;
int minzoom = 9;
if (centerlineGeometry != null) {
// prefer lake centerline if it exists
feature = features.geometry(LAYER_NAME, centerlineGeometry)
.setMinPixelSizeBelowZoom(13, 6d * element.name().length());
} else {
// otherwise just use a label point inside the lake
feature = features.pointOnSurface(LAYER_NAME);
Geometry geometry = element.source().worldGeometry();
double area = geometry.getArea();
minzoom = (int) Math.floor(20 - Math.log(area / WORLD_AREA_FOR_70K_SQUARE_METERS) / LOG2);
minzoom = Math.min(14, Math.max(9, minzoom));
}
feature
.setAttr(Fields.CLASS, FieldValues.CLASS_LAKE)
.setBufferPixels(BUFFER_SIZE)
.putAttrs(OmtLanguageUtils.getNames(element.source().tags(), translations))
.setAttr(Fields.INTERMITTENT, element.isIntermittent() ? 1 : 0)
.setMinZoom(minzoom);
} catch (GeometryException e) {
e.log(stats, "omt_water_polygon", "Unable to get geometry for water polygon " + element.source().id());
Geometry centerlineGeometry = lakeCenterlines.get(element.source().id());
FeatureCollector.Feature feature;
int minzoom = 9;
String place = element.place();
String clazz;
if ("bay".equals(element.natural())) {
clazz = FieldValues.CLASS_BAY;
} else if ("sea".equals(place)) {
clazz = FieldValues.CLASS_SEA;
} else {
clazz = FieldValues.CLASS_LAKE;
minzoom = 3;
}
if (centerlineGeometry != null) {
// prefer lake centerline if it exists
feature = features.geometry(LAYER_NAME, centerlineGeometry)
.setMinPixelSizeBelowZoom(13, 6d * element.name().length());
} else {
// otherwise just use a label point inside the lake
feature = features.pointOnSurface(LAYER_NAME)
.setMinZoom(place != null && SEA_OR_OCEAN_PLACE.contains(place) ? 0 : 3)
.setMinPixelSize(128); // tiles are 256x256, so 128x128 is 1/4 of a tile
}
feature
.setAttr(Fields.CLASS, clazz)
.setBufferPixels(BUFFER_SIZE)
.putAttrs(OmtLanguageUtils.getNames(element.source().tags(), translations))
.setAttr(Fields.INTERMITTENT, element.isIntermittent() ? 1 : 0)
.setMinZoom(minzoom);
}
}
}