diff --git a/CHANGELOG b/CHANGELOG index d0a658b..783d00b 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,5 +1,18 @@ ### Unreleased +- Keep counties out of the dominant-city rollup. A parent geohash cell rolled + up to the place that dominates its children on population, and `county` + counted as a city-like placetype, so a metro could be labelled with its + county's name while the smaller localities under it lost their own cells + (Brockport and Greece, NY answering "Monroe"). The rollup now only hands a + parent cell to a placetype listed in `--dominant-city-placetypes` + (default `locality,localadmin`, env `WOF_DOMINANT_CITY_PLACETYPES`); add + `county` to that list to reproduce earlier builds. Counties still win cells + through the normal comparator. Both routes into a parent cell -- winning the + dominant-city competition, and being the only city-like owner in the cell -- + share one eligibility gate, so a county cannot take a parent cell through the + single-candidate path either. + - Document the database backwards-compatibility contract in COMPATIBILITY.md and enforce it with permanent per-generation reader fixtures in spec/reader_compatibility_spec.js diff --git a/README.md b/README.md index 32de243..5d89902 100644 --- a/README.md +++ b/README.md @@ -223,8 +223,8 @@ The builder uses a multi-stage pipeline to decide which localities make it into 2. **Isolation pass** (`--isolation-min-population`): localities between the isolation floor and the primary threshold are evaluated as candidates. A candidate is promoted if at least one of its geohash cover cells (at base precision) is not already claimed by a primary locality. This ensures small but geographically isolated places like islands, remote towns, and oases get their own label without adding noise in dense urban areas. 3. **Country guarantee** (`--ensure-country-locality`): after the isolation pass, any country that still has zero localities gets its highest-population candidate promoted unconditionally. 4. **Contained-locality pruning** (`--drop-contained-localities`): removes localities whose polygon is fully contained inside a larger locality in the same country/admin1 group. -5. **Dominant-city rollup**: in the geohash index, when a major city (population >= `--dominant-locality-population`) dominates its neighbours by a ratio of `--dominant-locality-ratio`, smaller nearby localities are absorbed into the major city label. -6. **Locality-over-region promotion**: when a locality and a region compete for the same parent geohash cell, the locality wins if it covers >= `--parent-locality-min-share` of child cells. +5. **Dominant-city rollup**: in the geohash index, when a major city (population >= `--dominant-locality-population`) dominates its neighbours by a ratio of `--dominant-locality-ratio`, smaller nearby localities are absorbed into the major city label. Only the placetypes listed in `--dominant-city-placetypes` (default `locality,localadmin`) may play that dominant role, so a metro is never named after the county it sits in; counties still own cells on their own merits. +6. **Locality-over-region promotion**: when a locality and a region compete for the same parent geohash cell, the locality wins if it covers >= `--parent-locality-min-share` of child cells and its placetype is listed in `--dominant-city-placetypes`. Both routes to a parent cell share that eligibility gate, so a lone county cannot take a parent cell that the dominant-city rollup would have refused it. Builder notes: @@ -242,6 +242,7 @@ Builder notes: - Dominant-city rollup keeps broad city labels sticky in mixed city/suburb cells unless there is competing major-city pressure: - `--dominant-locality-population` (default `100000`) - `--dominant-locality-ratio` (default `3`) + - `--dominant-city-placetypes` (default `locality,localadmin`) placetypes eligible to be the dominant city of a parent cell, whether it wins a competition or is the only city-like owner in that cell. A county wins cells like any other place, but naming a whole parent cell after it reads as a mistake, so it is excluded by default; add `county` to reproduce pre-1.1 builds - Parent-cell takeover guard: - `--parent-locality-min-share` (default `0.5`) requires locality ownership of at least that child-cell share before replacing a parent cell label - Excludes neighbourhood-like placetypes from default reverse output @@ -282,6 +283,7 @@ Useful WOF build env vars: - `WOF_PROMOTE_LOCALITY_OVER_REGION=1|0` prefer locality labels over region in shared parent cells (default `1`) - `WOF_DOMINANT_LOCALITY_POPULATION` major-locality threshold for dominant-city rollup (default `100000`) - `WOF_DOMINANT_LOCALITY_RATIO` dominant-vs-next locality population ratio (default `3`) +- `WOF_DOMINANT_CITY_PLACETYPES` placetypes eligible to be a parent cell's dominant city (default `locality,localadmin`) - `WOF_PARENT_LOCALITY_MIN_SHARE` minimum child-cell share for locality parent takeover (default `0.5`) - `WOF_GEOMETRY_DECIMALS` round coordinates before storage/indexing (for example `4`) - `WOF_MIN_POPULATION` filter out places below threshold (for example `10000`) diff --git a/scripts/generate_boundary_index.js b/scripts/generate_boundary_index.js index 0737731..325f6f2 100755 --- a/scripts/generate_boundary_index.js +++ b/scripts/generate_boundary_index.js @@ -22,6 +22,11 @@ const PLACETYPE_BY_CODE = { 3: 'county' } +// Placetypes allowed to play the "dominant city" role in the parent-cell +// rollup. A county is a city-like placetype for ownership purposes, but naming +// a metro after its county reads as a mistake, so counties are out by default. +const DEFAULT_DOMINANT_CITY_PLACETYPES = ['locality', 'localadmin'] + function parseBool(value, defaultValue) { if (value === undefined || value === null || value === '') { return defaultValue @@ -38,6 +43,34 @@ function parseBool(value, defaultValue) { return defaultValue } +function parsePlacetypeList(value, defaultValue) { + if (value === undefined || value === null || String(value).trim() === '') { + return defaultValue + } + + var parts = String(value).split(',') + var placetypes = [] + for (var i = 0; i < parts.length; i++) { + var name = parts[i].toLowerCase().trim() + if (!name) continue + if (!Object.prototype.hasOwnProperty.call(PLACETYPE_CODES, name)) { + throw new Error('Unknown placetype in --dominant-city-placetypes: ' + parts[i].trim()) + } + if (!isCityPlacetypeCode(PLACETYPE_CODES[name])) { + throw new Error('--dominant-city-placetypes only accepts city-like placetypes: ' + parts[i].trim()) + } + if (placetypes.indexOf(name) === -1) { + placetypes.push(name) + } + } + + if (!placetypes.length) { + throw new Error('--dominant-city-placetypes requires at least one placetype') + } + + return placetypes +} + function parseArgs(argv) { var opts = { database: null, @@ -68,7 +101,8 @@ function parseArgs(argv) { promoteLocalityOverRegion: true, dominantLocalityPopulation: 100000, dominantLocalityRatio: 3, - parentLocalityMinShare: 0.5 + parentLocalityMinShare: 0.5, + dominantCityPlacetypes: DEFAULT_DOMINANT_CITY_PLACETYPES.slice() } for (var i = 0; i < argv.length; i++) { @@ -142,6 +176,8 @@ function parseArgs(argv) { } else if (arg === '--dominant-locality-ratio') { var dominantRatio = Number(argv[++i]) opts.dominantLocalityRatio = Number.isFinite(dominantRatio) ? dominantRatio : opts.dominantLocalityRatio + } else if (arg === '--dominant-city-placetypes') { + opts.dominantCityPlacetypes = parsePlacetypeList(argv[++i], opts.dominantCityPlacetypes) } else if (arg === '--parent-locality-min-share') { var minShare = Number(argv[++i]) opts.parentLocalityMinShare = Number.isFinite(minShare) ? minShare : opts.parentLocalityMinShare @@ -194,6 +230,7 @@ function usage() { ' --dominant-locality-population Population threshold that marks locality as major for dominant-city rollups (default: 100000)', ' --dominant-locality-ratio Required dominant-vs-next population ratio for locality rollups (default: 3)', ' --parent-locality-min-share Minimum child-cell share (0..1) required to let a locality take over a parent cell (default: 0.5)', + ' --dominant-city-placetypes Comma-separated placetypes eligible to take over a parent cell, by competition or as its only city-like owner (default: ' + DEFAULT_DOMINANT_CITY_PLACETYPES.join(',') + ')', ' --append Keep existing boundary rows and append/replace by place id', ' --replace Clear boundary rows first (default)', ' --help, -h Show this help message' @@ -946,6 +983,28 @@ function isCityPlacetypeCode(code) { return code === PLACETYPE_CODES.locality || code === PLACETYPE_CODES.localadmin || code === PLACETYPE_CODES.county } +// Owning a cell and standing in for a whole metro are different jobs. Every +// city-like placetype (county included) can win a cell through the comparator; +// only these placetypes may become the dominant city a parent cell is named +// after, so a metro never ends up labelled with its county's name. +function isDominantCityPlacetypeCode(code, opts) { + if (!isCityPlacetypeCode(code)) { + return false + } + + var names = opts && Array.isArray(opts.dominantCityPlacetypes) && opts.dominantCityPlacetypes.length + ? opts.dominantCityPlacetypes + : DEFAULT_DOMINANT_CITY_PLACETYPES + + for (var i = 0; i < names.length; i++) { + if (PLACETYPE_CODES[names[i]] === code) { + return true + } + } + + return false +} + // Rebuild a comparable place record from a stored compact_places row so that // appended batches can be ranked against places written by earlier batches. // Databases created before population/area were stored yield NULL for those @@ -1011,6 +1070,7 @@ function selectDominantLocalityId(localityIds, placeById, opts) { var place = placeById[String(id)] return { id: Number(id), + placetypeCode: place ? place.placetypeCode : null, population: placePopulation(place) } }) @@ -1065,6 +1125,24 @@ function localityShareMeetsThreshold(localityId, group, opts) { return localityShareInParent(localityId, group) >= threshold } +// The single gate for taking over a parent cell, shared by both roll-up paths +// -- the lone city-like owner and the winner of a dominant-city competition -- +// so that neither can drift away from the other. A candidate qualifies only if +// its placetype may name a parent cell (counties may not, by default) and it +// owns enough of the parent's child cells to stand in for the whole of it. +// A candidate rejected here is not replaced by a runner-up: the runner-up lost +// the cell competition, so promoting it would name the parent after a place +// that owns less of it. The parent keeps its existing owner instead, and every +// child keeps the cell it won. +function localityMayTakeOverParent(localityId, group, placeById, opts) { + var place = placeById[String(localityId)] + if (!place || !isDominantCityPlacetypeCode(place.placetypeCode, opts)) { + return false + } + + return localityShareMeetsThreshold(localityId, group, opts) +} + function promoteLocalityParentsByRegionCompetition(bestByHash, placeById, opts) { if (!opts.promoteLocalityOverRegion) { return @@ -1123,7 +1201,7 @@ function promoteLocalityParentsByRegionCompetition(bestByHash, placeById, opts) if (localityIds.length === 1) { var localityId = Number(localityIds[0]) - if (!localityShareMeetsThreshold(localityId, group, opts)) { + if (!localityMayTakeOverParent(localityId, group, placeById, opts)) { continue } @@ -1148,7 +1226,7 @@ function promoteLocalityParentsByRegionCompetition(bestByHash, placeById, opts) if (dominantLocalityId === null) { continue } - if (!localityShareMeetsThreshold(dominantLocalityId, group, opts)) { + if (!localityMayTakeOverParent(dominantLocalityId, group, placeById, opts)) { continue } @@ -1865,6 +1943,9 @@ async function main() { if (!Number.isFinite(options.dominantLocalityRatio) || options.dominantLocalityRatio < 1) { options.dominantLocalityRatio = 1 } + if (!Array.isArray(options.dominantCityPlacetypes) || !options.dominantCityPlacetypes.length) { + options.dominantCityPlacetypes = DEFAULT_DOMINANT_CITY_PLACETYPES.slice() + } if (!Number.isFinite(options.parentLocalityMinShare)) { options.parentLocalityMinShare = 0.5 } @@ -1963,7 +2044,7 @@ async function main() { console.log('Dense county rule: area_km2<=' + options.countyDenseMaxAreaKm2 + ' => max_precision=' + options.countyDenseMaxPrecision) } if (options.dominantLocalityPopulation > 0) { - console.log('Dominant locality rollup: major_population>=' + options.dominantLocalityPopulation + ', ratio>=' + options.dominantLocalityRatio) + console.log('Dominant locality rollup: major_population>=' + options.dominantLocalityPopulation + ', ratio>=' + options.dominantLocalityRatio + ', placetypes=' + options.dominantCityPlacetypes.join(',')) } else { console.log('Dominant locality rollup: disabled') } diff --git a/scripts/generate_wof_boundary.sh b/scripts/generate_wof_boundary.sh index f98ce3e..ee426d7 100755 --- a/scripts/generate_wof_boundary.sh +++ b/scripts/generate_wof_boundary.sh @@ -29,6 +29,7 @@ set -euo pipefail # WOF_DOMINANT_LOCALITY_POPULATION Major-locality threshold for dominant-city rollup (default: 100000) # WOF_DOMINANT_LOCALITY_RATIO Dominant-vs-next locality population ratio (default: 3) # WOF_PARENT_LOCALITY_MIN_SHARE Minimum child-cell share (0..1) required for locality parent takeover (default: 0.5) +# WOF_DOMINANT_CITY_PLACETYPES Placetypes eligible to take over a parent cell (default: locality,localadmin) # WOF_INCLUDE_LOCALADMIN Include localadmin placetypes (default: 0) # WOF_INCLUDE_COUNTY Include county placetypes (default: 1) # WOF_INCLUDE_REGION Include region placetypes (default: 1) @@ -72,6 +73,7 @@ WOF_PROMOTE_LOCALITY_OVER_REGION="${WOF_PROMOTE_LOCALITY_OVER_REGION:-1}" WOF_DOMINANT_LOCALITY_POPULATION="${WOF_DOMINANT_LOCALITY_POPULATION:-100000}" WOF_DOMINANT_LOCALITY_RATIO="${WOF_DOMINANT_LOCALITY_RATIO:-3}" WOF_PARENT_LOCALITY_MIN_SHARE="${WOF_PARENT_LOCALITY_MIN_SHARE:-0.5}" +WOF_DOMINANT_CITY_PLACETYPES="${WOF_DOMINANT_CITY_PLACETYPES:-locality,localadmin}" WOF_INCLUDE_LOCALADMIN="${WOF_INCLUDE_LOCALADMIN:-0}" WOF_INCLUDE_COUNTY="${WOF_INCLUDE_COUNTY:-1}" WOF_INCLUDE_REGION="${WOF_INCLUDE_REGION:-1}" @@ -151,6 +153,7 @@ COMMON_FLAGS=( --dominant-locality-population "${WOF_DOMINANT_LOCALITY_POPULATION}" --dominant-locality-ratio "${WOF_DOMINANT_LOCALITY_RATIO}" --parent-locality-min-share "${WOF_PARENT_LOCALITY_MIN_SHARE}" + --dominant-city-placetypes "${WOF_DOMINANT_CITY_PLACETYPES}" --include-localadmin "${WOF_INCLUDE_LOCALADMIN}" --include-county "${WOF_INCLUDE_COUNTY}" --include-region "${WOF_INCLUDE_REGION}" diff --git a/spec/boundary_builder_spec.js b/spec/boundary_builder_spec.js index d788e76..83ac071 100644 --- a/spec/boundary_builder_spec.js +++ b/spec/boundary_builder_spec.js @@ -17,6 +17,107 @@ function close(db) { }); } +const DOMINANT_COUNTY_PARENT_HASH = 's000'; + +function rectangleRing(minLon, minLat, maxLon, maxLat) { + return [[ + [minLon, minLat], + [maxLon, minLat], + [maxLon, maxLat], + [minLon, maxLat], + [minLon, minLat] + ]]; +} + +function polygonFeature(id, name, placetype, population, ring) { + const properties = { + name, + placetype, + country_id: 'US', + admin1_id: 36, + is_current: 1 + }; + + if (population !== null) { + properties.population = population; + } + + return { + type: 'Feature', + id, + properties, + geometry: { type: 'Polygon', coordinates: ring } + }; +} + +// Covers the whole parent cell, so it owns that cell outright and stands in as +// the fallback label the rollup competes against. +function regionFeature(id, name, bbox) { + return polygonFeature(id, name, 'region', null, rectangleRing(bbox.minLon, bbox.minLat, bbox.maxLon, bbox.maxLat)); +} + +// A town sitting inside one child cell, inset so that it never touches a +// neighbouring cell: a polygon that shares an edge with the next cell claims +// that cell too, which would put both towns in competition for the same cells. +function childCellFeature(id, name, placetype, population, childHash) { + const cell = geohash.decodeBbox(childHash); + const insetLon = (cell.maxLon - cell.minLon) / 4; + const insetLat = (cell.maxLat - cell.minLat) / 4; + + return polygonFeature(id, name, placetype, population, rectangleRing( + cell.minLon + insetLon, + cell.minLat + insetLat, + cell.maxLon - insetLon, + cell.maxLat - insetLat + )); +} + +function descendantCountSql(parentHash, placeId) { + return `SELECT COUNT(*) AS count FROM compact_geohash_lookup WHERE geohash LIKE '${parentHash}%' AND geohash <> '${parentHash}' AND place_id = ${placeId}`; +} + +// The Rochester shape: a county that outpopulates every town inside it, two +// small towns that each own their own child cells, and a region as fallback. +function writeDominantCountyFixture(inputPath) { + const bbox = geohash.decodeBbox(DOMINANT_COUNTY_PARENT_HASH); + const height = bbox.maxLat - bbox.minLat; + const children = geohash.children(DOMINANT_COUNTY_PARENT_HASH); + + fs.writeFileSync(inputPath, JSON.stringify({ + type: 'FeatureCollection', + features: [ + regionFeature(8001, 'Fallback Region', bbox), + // Stops short of the cell's northern edge, the way a real county + // boundary does, so it owns the child cells and not the parent itself. + polygonFeature(8002, 'Monroe County', 'county', 748482, + rectangleRing(bbox.minLon, bbox.minLat, bbox.maxLon, bbox.maxLat - (height / 100))), + childCellFeature(8003, 'Brockport', 'locality', 8366, children[0]), + childCellFeature(8004, 'Greece', 'locality', 14519, children[20]) + ] + })); + + return inputPath; +} + +// The same shape with the towns taken out: the county is then the only +// city-like owner inside the parent cell, which sends the roll-up down its +// single-candidate path instead of the dominant-city competition. +function writeLoneCountyFixture(inputPath) { + const bbox = geohash.decodeBbox(DOMINANT_COUNTY_PARENT_HASH); + const height = bbox.maxLat - bbox.minLat; + + fs.writeFileSync(inputPath, JSON.stringify({ + type: 'FeatureCollection', + features: [ + regionFeature(8101, 'Fallback Region', bbox), + polygonFeature(8102, 'Monroe County', 'county', 748482, + rectangleRing(bbox.minLon, bbox.minLat, bbox.maxLon, bbox.maxLat - (height / 100))) + ] + })); + + return inputPath; +} + describe('boundary builder', () => { it('drops contained localities when pruning is enabled', async () => { const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'offline-geocoder-builder-')); @@ -645,4 +746,250 @@ describe('boundary builder', () => { fs.rmSync(dir, { recursive: true, force: true }); } }); + it('does not roll a parent cell up to a dominant county', async () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'offline-geocoder-builder-')); + try { + const inputPath = path.join(dir, 'dominant-county.geojson'); + const dbPath = path.join(dir, 'dominant-county.sqlite'); + + const result = spawnSync('node', [ + path.join(__dirname, '..', 'scripts', 'generate_boundary_index.js'), + '--database', dbPath, + '--input', writeDominantCountyFixture(inputPath), + '--index-mode', 'compact', + '--include-region', 'true', + '--include-county', 'true', + '--base-precision', '4', + '--max-precision', '5', + '--dominant-locality-population', '100000', + '--dominant-locality-ratio', '3' + ], { encoding: 'utf8' }); + + expect(result.status).toEqual(0); + + const db = new sqlite3.Database(dbPath); + try { + // The county outranks every town on population, so it must not carry + // the parent cell -- and the runner-up town must not inherit the + // rollup either, because it lost the same competition. + const parentRow = await all(db, `SELECT geohash, place_id FROM compact_geohash_lookup WHERE geohash='${DOMINANT_COUNTY_PARENT_HASH}'`); + expect(parentRow).toEqual([{ geohash: DOMINANT_COUNTY_PARENT_HASH, place_id: 8001 }]); + + const smallTownCells = await all(db, descendantCountSql(DOMINANT_COUNTY_PARENT_HASH, 8003)); + expect(smallTownCells[0].count).toBeGreaterThan(0); + + const largerTownCells = await all(db, descendantCountSql(DOMINANT_COUNTY_PARENT_HASH, 8004)); + expect(largerTownCells[0].count).toBeGreaterThan(0); + } finally { + await close(db); + } + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } + }); + + it('rolls a parent cell up to a dominant county when county is opted back in', async () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'offline-geocoder-builder-')); + try { + const inputPath = path.join(dir, 'dominant-county-opt-in.geojson'); + const dbPath = path.join(dir, 'dominant-county-opt-in.sqlite'); + + const result = spawnSync('node', [ + path.join(__dirname, '..', 'scripts', 'generate_boundary_index.js'), + '--database', dbPath, + '--input', writeDominantCountyFixture(inputPath), + '--index-mode', 'compact', + '--include-region', 'true', + '--include-county', 'true', + '--base-precision', '4', + '--max-precision', '5', + '--dominant-locality-population', '100000', + '--dominant-locality-ratio', '3', + '--dominant-city-placetypes', 'locality,localadmin,county' + ], { encoding: 'utf8' }); + + expect(result.status).toEqual(0); + + const db = new sqlite3.Database(dbPath); + try { + const parentRow = await all(db, `SELECT geohash, place_id FROM compact_geohash_lookup WHERE geohash='${DOMINANT_COUNTY_PARENT_HASH}'`); + expect(parentRow).toEqual([{ geohash: DOMINANT_COUNTY_PARENT_HASH, place_id: 8002 }]); + + const smallTownCells = await all(db, descendantCountSql(DOMINANT_COUNTY_PARENT_HASH, 8003)); + expect(smallTownCells[0].count).toEqual(0); + + const largerTownCells = await all(db, descendantCountSql(DOMINANT_COUNTY_PARENT_HASH, 8004)); + expect(largerTownCells[0].count).toEqual(0); + } finally { + await close(db); + } + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } + }); + + it('does not roll a parent cell up to a county that is its only city-like owner', async () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'offline-geocoder-builder-')); + try { + const inputPath = path.join(dir, 'lone-county.geojson'); + const dbPath = path.join(dir, 'lone-county.sqlite'); + + const result = spawnSync('node', [ + path.join(__dirname, '..', 'scripts', 'generate_boundary_index.js'), + '--database', dbPath, + '--input', writeLoneCountyFixture(inputPath), + '--index-mode', 'compact', + '--include-region', 'true', + '--include-county', 'true', + '--base-precision', '4', + '--max-precision', '5' + ], { encoding: 'utf8' }); + + expect(result.status).toEqual(0); + + const db = new sqlite3.Database(dbPath); + try { + // The county is the sole city-like child owner and covers the whole + // parent cell, so it clears the child-share threshold -- but taking + // the parent cell over would still label the metro with the county's + // name, which is what the placetype gate exists to prevent. + const parentRow = await all(db, `SELECT geohash, place_id FROM compact_geohash_lookup WHERE geohash='${DOMINANT_COUNTY_PARENT_HASH}'`); + expect(parentRow).toEqual([{ geohash: DOMINANT_COUNTY_PARENT_HASH, place_id: 8101 }]); + + // Losing the roll-up costs the county nothing below the parent cell. + const countyCells = await all(db, descendantCountSql(DOMINANT_COUNTY_PARENT_HASH, 8102)); + expect(countyCells[0].count).toBeGreaterThan(0); + } finally { + await close(db); + } + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } + }); + + it('rolls a parent cell up to a lone county when county is opted back in', async () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'offline-geocoder-builder-')); + try { + const inputPath = path.join(dir, 'lone-county-opt-in.geojson'); + const dbPath = path.join(dir, 'lone-county-opt-in.sqlite'); + + const result = spawnSync('node', [ + path.join(__dirname, '..', 'scripts', 'generate_boundary_index.js'), + '--database', dbPath, + '--input', writeLoneCountyFixture(inputPath), + '--index-mode', 'compact', + '--include-region', 'true', + '--include-county', 'true', + '--base-precision', '4', + '--max-precision', '5', + '--dominant-city-placetypes', 'locality,localadmin,county' + ], { encoding: 'utf8' }); + + expect(result.status).toEqual(0); + + const db = new sqlite3.Database(dbPath); + try { + const parentRow = await all(db, `SELECT geohash, place_id FROM compact_geohash_lookup WHERE geohash='${DOMINANT_COUNTY_PARENT_HASH}'`); + expect(parentRow).toEqual([{ geohash: DOMINANT_COUNTY_PARENT_HASH, place_id: 8102 }]); + } finally { + await close(db); + } + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } + }); + + it('still rolls a parent cell up to a dominant city when a county owns cells beside it', async () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'offline-geocoder-builder-')); + try { + const inputPath = path.join(dir, 'dominant-city-with-county.geojson'); + const parentHash = 's000'; + const parentBbox = geohash.decodeBbox(parentHash); + const width = parentBbox.maxLon - parentBbox.minLon; + const height = parentBbox.maxLat - parentBbox.minLat; + // The eastern quarter of the parent cell is left to the hamlet, the + // county and the region; the metro takes everything west of it. + const metroMaxLon = parentBbox.minLon + (width * 0.75) - (width / 1000); + const eastern = geohash.children(parentHash) + .filter((child) => geohash.decodeBbox(child).minLon > metroMaxLon); + + fs.writeFileSync(inputPath, JSON.stringify({ + type: 'FeatureCollection', + features: [ + regionFeature(9001, 'Fallback Region', parentBbox), + // Sprawls over the parent cell without filling it, so it owns child + // cells rather than the parent cell itself. + polygonFeature(9002, 'Metro Core', 'locality', 1200000, + rectangleRing(parentBbox.minLon, parentBbox.minLat, metroMaxLon, parentBbox.maxLat - (height / 100))), + childCellFeature(9003, 'Rural Hamlet', 'locality', 18000, eastern[0]), + childCellFeature(9004, 'Rural County', 'county', 30000, eastern[eastern.length - 1]) + ] + })); + + const build = (dbPath, dominantPopulation) => spawnSync('node', [ + path.join(__dirname, '..', 'scripts', 'generate_boundary_index.js'), + '--database', dbPath, + '--input', inputPath, + '--index-mode', 'compact', + '--include-region', 'true', + '--include-county', 'true', + '--base-precision', '4', + '--max-precision', '5', + '--dominant-locality-population', dominantPopulation, + '--dominant-locality-ratio', '3' + ], { encoding: 'utf8' }); + + // With the rollup out of reach, the hamlet and the county each hold a + // child cell of their own: the rollup below has something to fold in. + const withoutRollupPath = path.join(dir, 'without-rollup.sqlite'); + expect(build(withoutRollupPath, '5000000').status).toEqual(0); + + const withoutRollup = new sqlite3.Database(withoutRollupPath); + try { + const hamletCells = await all(withoutRollup, descendantCountSql(parentHash, 9003)); + expect(hamletCells[0].count).toBeGreaterThan(0); + + const countyCells = await all(withoutRollup, descendantCountSql(parentHash, 9004)); + expect(countyCells[0].count).toBeGreaterThan(0); + } finally { + await close(withoutRollup); + } + + const dbPath = path.join(dir, 'dominant-city-with-county.sqlite'); + expect(build(dbPath, '100000').status).toEqual(0); + + const db = new sqlite3.Database(dbPath); + try { + const parentRow = await all(db, `SELECT geohash, place_id FROM compact_geohash_lookup WHERE geohash='${parentHash}'`); + expect(parentRow).toEqual([{ geohash: parentHash, place_id: 9002 }]); + + const hamletCells = await all(db, descendantCountSql(parentHash, 9003)); + expect(hamletCells[0].count).toEqual(0); + } finally { + await close(db); + } + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } + }); + it('rejects a dominant-city placetype that cannot own a city label', async () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'offline-geocoder-builder-')); + try { + const inputPath = path.join(dir, 'dominant-city-placetypes.geojson'); + const dbPath = path.join(dir, 'dominant-city-placetypes.sqlite'); + + const result = spawnSync('node', [ + path.join(__dirname, '..', 'scripts', 'generate_boundary_index.js'), + '--database', dbPath, + '--input', writeDominantCountyFixture(inputPath), + '--index-mode', 'compact', + '--dominant-city-placetypes', 'locality,region' + ], { encoding: 'utf8' }); + + expect(result.status).not.toEqual(0); + expect(result.stderr).toContain('--dominant-city-placetypes only accepts city-like placetypes'); + } finally { + fs.rmSync(dir, { recursive: true, force: true }); + } + }); });