Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add turf-voronoi module for #1038 #1043

Merged
merged 18 commits into from
Oct 27, 2017
Merged

Conversation

stevage
Copy link
Collaborator

@stevage stevage commented Oct 26, 2017

  • Use a meaningful title for the pull request. Include the name of the package modified.
  • Have read How To Contribute.
  • Run npm test at the sub modules where changes have occurred.
  • Run npm run lint to ensure code style at the turf module level.

npm run lint fails with this helpful message:

9:1 error JSDoc syntax error valid-jsdoc

I take it I need to include some invalid jsdoc to pass the lint?

Submitting a new TurfJS Module.

  • Overview description of proposed module.
  • Include JSDocs with a basic example.
  • Execute ./scripts/generate-readmes to create README.md.
  • Add yourself to contributors in package.json using "Full Name <@github Username>".

import { lineString } from '@turf/helpers';
import d3voronoi from 'd3-voronoi';

function und3ify(polygon) {
Copy link
Collaborator

@stebogit stebogit Oct 26, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stevage all the functions need a JDocs block, I bet that's why probably you're getting an error

*
* The Voronoi algorithim used comes from the d3-voronoi package.
*
* @name voronoi
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stevage this should be turfVoronoi

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure about this? I've made this change, but Travis is failing and saying "not ok 12 voronoi is missing from index.js". Every other module I've looked at uses a short name that matches the function name.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stevage I believe it's referring to the main @turf/turf index.js, might be wrong though

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@name voronoi is good.

turfVoronoi is incorrect.

* var voronoiPolygons = turf.voronoi(points, bbox);
*/
function voronoi(points, bbox) {
if (!points) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stevage maybe we should instead throw an error here (since the parameter is required) or at least return "something", so it's clear that the input was empty.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, both points & bbox should return Errors if not provided.

if (!points) throw new Error('points is required');
if (!bbox) throw new Error('bbox is required');


var polygonsd3 = d3v(pointsd3).polygons();

return {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stevage you could import featureCollection from helpers and:

return featureCollection(polygonsd3.map(und3ify));

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

* The Voronoi algorithim used comes from the d3-voronoi package.
*
* @name voronoi
* @param {FeatureCollection<Point>|Feature[]<Point>} points to find the Voronoi polygons around.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The input should only be FeatureCollection<Point>, an Array of Features isn't a valid GeoJSON and a single point doesn't would most likely not be valid for this type of operation.

* @name voronoi
* @param {FeatureCollection<Point>|Feature[]<Point>} points to find the Voronoi polygons around.
* @param {number[]} bbox clipping rectangle, in [minX, minY, maxX, MaxY] order.
* @returns {FeatureCollection<Polygon} a set of polygons, one per input polygon.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing a closing bracket, that's most likely the issue with the JSDocs.

FeatureCollection<Polygon => FeatureCollection<Polygon>

* @returns {FeatureCollection<Polygon} a set of polygons, one per input polygon.
* @example
* var bbox = [-70, 40, -60, 60];
* var points = turf.random('points', 100, { bbox: bbox });
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The random module has changed in the newest release.

var points = turf.randomPoints(100, { bbox: bbox }); 

@stebogit I guess we could re-implement the random() method with the legacy behavior.

* var voronoiPolygons = turf.voronoi(points, bbox);
*/
function voronoi(points, bbox) {
if (!points) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, both points & bbox should return Errors if not provided.

if (!points) throw new Error('points is required');
if (!bbox) throw new Error('bbox is required');

if (!points) {
return;
}
var features = Array.isArray(points) ? points : points.features;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Array of Features shouldn't be handled since it's not a valid GeoJSON input.


var polygonsd3 = d3v(pointsd3).polygons();

return {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@@ -0,0 +1,51 @@
{
"name": "@turf/voronoi",
"version": "0.1.0",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The version can be 5.0.0 since this will be published under that major release.

{
"name": "@turf/voronoi",
"version": "0.1.0",
"description": "turf module to generate Voronoi polygons from a set of points and a bounding box",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are using JSDocs for the description of the modules, since no one will be maintaining the package.json we just define this description as very generic.

"description": "turf voronoi module",

"module": "index",
"jsnext:main": "index",
"files": [
"index.js"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll also need to provide a Typescript definition, index.d.ts and the main.js (Rollup build output).
Look at any other TurfJS package.json for examples.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, added - but I have no idea what I'm doing here, or how this file is used. This would be a good thing to document in CONTRIBUTING.md.

"dependencies": {
"@turf/linestring-to-polygon": "^5.0.0",
"d3-voronoi": "^1.1.2",
"turf-helpers": "^3.x"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use the namespace turf "@turf/helpers": "5.x"

@DenisCarriere
Copy link
Member

👍 This module will look pretty cool!

image

@stevage
Copy link
Collaborator Author

stevage commented Oct 26, 2017

Thanks very much for the comments and suggestions above. I think I implemented them all.

Just a thought: an alternative way to express the body of this function is in functional style:

    return featureCollection(
                d3voronoi.voronoi()
                    .extent([[bbox[0], bbox[1]], [bbox[2], bbox[3]]])(points.features.map(getCoords))
                    .polygons()
                    .map(coordsToPolygon)
            );

Any preferences one way or the other?

@stebogit
Copy link
Collaborator

@stevage I believe eslint will tell you 😄
However I haven't seen using that notation so far in Turf, but I might be wrong

@DenisCarriere
Copy link
Member

DenisCarriere commented Oct 27, 2017

@stevage Functional style works, you can drop the .map() by using getCoords().

Edit: You're example did use getCoords 🤦‍♂️

    return featureCollection(
                d3voronoi.voronoi()
                    .extent([[bbox[0], bbox[1]], [bbox[2], bbox[3]]])(points.features.map(getCoords))
                    .polygons()
                    .map(coordsToPolygon)
            );

/**
* http://turfjs.org/docs/#voronoi
*/
export default function voronoi(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Look great! Don't have to change a thing

These TurfJS typescript definitions are getting a lot easier to write as we go.

Copy link
Member

@DenisCarriere DenisCarriere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 published under @turf/voronoi

@DenisCarriere
Copy link
Member

DenisCarriere commented Oct 27, 2017

❌ Error

d3-voronoi doesn't have a default export, you will have to add a * before the import.

Could be fixed on d3-voronoi's side, but for now lets just add * or import {voronoi}

lerna ERR! test Errored while running script in '@turf/turf'
lerna ERR! execute Error: Command failed: npm run test
lerna ERR! execute 
lerna ERR! execute index.js → turf.js...
lerna ERR! execute [!] Error: 'default' is not exported by ../turf-voronoi/node_modules/d3-voronoi/index.js
lerna ERR! execute https:/rollup/rollup/wiki/Troubleshooting#name-is-not-exported-by-module
lerna ERR! execute ../turf-voronoi/index.js (4:7)
lerna ERR! execute 2: import { lineString, featureCollection } from '@turf/helpers';
lerna ERR! execute 3: import { getCoords, collectionOf } from '@turf/invariant';
lerna ERR! execute 4: import d3voronoi from 'd3-voronoi';

You can also test this by not doing the Rollup build and only using @std/esm.

$ rm main.js
$ node -r @std/esm test.js
SyntaxError: Module '/Users/mac/Github/turf-voronoi/packages/turf-voronoi/node_modules/d3-voronoi/build/d3-voronoi.js' does not provide an export named 'default'
    at Object.<anonymous> (/Users/mac/Github/turf-voronoi/packages/turf-voronoi/index.js)

@stevage
Copy link
Collaborator Author

stevage commented Oct 27, 2017

Oh...bugger. I was just making all these same changes at my end, heh.

@DenisCarriere
Copy link
Member

Oh...bugger. I was just making all these same changes at my end, heh.

Lol 😄 I didn't do that much, mostly published the module, that was the biggest thing.

@stevage
Copy link
Collaborator Author

stevage commented Oct 27, 2017

Heh. Looks like there are still a lot of undocumented steps in the CONTRIBUTING.md, like adding the package to turf/index.js, turf/index.d.ts, turf/package.json etc.

@DenisCarriere
Copy link
Member

DenisCarriere commented Oct 27, 2017

@stevage Well you can't add index.js unless the module is published, and you can't publish unless you have publishing rights... so this would only apply to new modules.

Right now the tests are pretty aggressive, they could be more Slack on the unpublished modules.

@stevage Feel free to add to the CONTRIBUTING guide :) It's been mostly the same people contributing new modules.

@DenisCarriere
Copy link
Member

❌ Un-related error

@stebogit This error keeps poping up a bunch of times, it comes from @turf/isobands's JSDoc @example running out of memory.

I wonder why this happens sporadically, maybe we could tone down the example for that module to prevent any un-related errors.

 # turf-example-isobands
lerna ERR! execute 
lerna ERR! execute <--- Last few GCs --->
lerna ERR! execute 
lerna ERR! execute [10543:0x2ca23e0]    18469 ms: Mark-sweep 1409.8 (1458.9) -> 1409.8 (1442.9) MB, 1266.0 / 0.0 ms  (+ 0.0 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 1266 ms) last resort GC in old space requested
lerna ERR! execute [10543:0x2ca23e0]    19737 ms: Mark-sweep 1409.8 (1442.9) -> 1409.8 (1442.9) MB, 1268.5 / 0.0 ms  last resort GC in old space requested
lerna ERR! execute 
lerna ERR! execute 
lerna ERR! execute <--- JS stacktrace --->
lerna ERR! execute 
lerna ERR! execute ==== JS stack trace =========================================
lerna ERR! execute 
lerna ERR! execute Security context: 0x74e4c9a5e91 <JSObject>
lerna ERR! execute     1: BandGrid2AreaPaths(aka BandGrid2AreaPaths) [/home/travis/build/Turfjs/turf/packages/turf/turf.js:~74222] [pc=0x25a08bebfcc0](this=0x234742d02311 <undefined>,grid=0x14368ca12061 <Object map = 0x16c979eace11>)
lerna ERR! execute     3: isoBands(aka isoBands) [/home/travis/build/Turfjs/turf/packages/turf/turf.js:72217] [bytecode=0x1c832defaba1 offset=296](this=0x234742d02311 <undefined>,data=0x14368ca12091 <JSA...

@DenisCarriere
Copy link
Member

@stevage 16 commits later... All checks passed! 😀

I'm ready to merge whenever you're good with it.

image

@stevage
Copy link
Collaborator Author

stevage commented Oct 27, 2017

Woo :)

Yeah, I'm good with it. Although I probably should have mentioned earlier that one downside of using D3 is that the calculations are completely 2D. I don't imagine it will materially matter for most applications, but close to the poles the polygons will be completely wrong.

@DenisCarriere
Copy link
Member

Totally not worried about the 3D stuff, barely any modules in turf handle that type of complexity.

Oh one more thing, could you add Mike as a contributor to this module since he pretty much did the 95% of the work to make this happen :)

@DenisCarriere DenisCarriere merged commit eb83168 into Turfjs:master Oct 27, 2017
@mbostock
Copy link

You might consider using Philippe Rivière’s d3-geo-voronoi instead of d3-voronoi for geographic polygons, or Loren Petrich's spherical Delaunay Triangulation library, upon which d3-geo-voronoi is based.

However, note that adopting a spherical Voronoi library introduces some subtle challenges when working with standard GeoJSON, since standard GeoJSON uses planar equirectangular coordinates rather than true spherical coordinates. (See the d3-geo README.)

Also, there’s no need to add me as a contributor just because you’re using d3-voronoi, though thank you. The d3-voronoi library is based largely on work by Raymond Hill and Steven Fortune. If I ever find the time, I am planning on rewriting d3-voronoi to use Vladimir Agafonkin’s delaunator, since it will be faster and more stable for certain numerical edge cases.

@DenisCarriere
Copy link
Member

DenisCarriere commented Oct 27, 2017

Wow, thanks for the feedback @mbostock! We can definitely switch over to d3-geo-voronoi instead.

Looking forward to the new and improved d3-voronoi using Vlad's delaunator library.

there’s no need to add me as a contributor just because you’re using d3-voronoi

We try to include dependency author's to the contributor list, as much as we can, it's the least we can do for including this awesome library 😄

@DenisCarriere
Copy link
Member

DenisCarriere commented Oct 29, 2017

I've tried to implement d3-geo-voronoi and it kept breaking, maybe it's me... but I couldn't get it to work with relative ease. We can put the d3-geo-voronoi replacement on the back burner for now.

CC: @mbostock @stevage @Fil

@Fil
Copy link

Fil commented Oct 29, 2017

The thing with geo-voronoi is that it uses the correct (geodesic) distance, and when dealing with Voronoi you probably want distances to be meaningful. Also, the results of a geometric operation should be independent of the reference rotation.

Please give an example of how d3-geo-voronoi "keeps breaking" and don't hesitate to open an issue as necessary on https:/Fil/d3-geo-voronoi; I'll try and help if I can. It's true that this code is quite unstable.

@DenisCarriere
Copy link
Member

@Fil Thanks for the feedback, I'll send some examples to d3-geo-voronoi, might just be me entering the method wrong.

@maral
Copy link
Contributor

maral commented Dec 30, 2023

Just to follow-up on the d3-geo-voronoi, I now tested all options - d3-geo-voronoi, d3-delaunay, and @turf/voronoi (still based on the deprecated d3-voronoi, and d3-geo-voronoi was the only inconsistent - whenever the points were closer together, the polygons were overlapping and very weird in general. It started working when every two points were ~1 km apart (but still was not perfect).

For now the best results I get when I convert the WGS-84 points to mercator projection, apply planar voronoi and then convert the coordinates back to WGS-84.

@Fil
Copy link

Fil commented Dec 30, 2023

Can you share an actual example? thanks

@maral
Copy link
Contributor

maral commented Dec 30, 2023

Hi @Fil, I am fan of your work, would love to see it working. Here is the data set I was testing it on: https://jsfiddle.net/h6qz279n/1/

The bugs look very similar to this issue. Let me know if I am doing something wrong.

@Fil
Copy link

Fil commented Dec 31, 2023

Leaflet doesn't support geodesic line interpolation, it just draws a straight line between the projected coordinates. In that sense it's not compatible with geoVoronoi which generates polygons that cover the whole sphere.

In this case I'd use a planar voronoi—maybe add 4 “corners” around the city. Note that on a small area the error is negligible https://observablehq.com/@visionscarto/planar-vs-spherical-voronoi

On top of that I think you are correct that this dataset also triggers numerical instability (Fil/d3-geo-voronoi#10).

@maral
Copy link
Contributor

maral commented Dec 31, 2023

Yes, I implemented the planar voronoi (via d3-delaunay) successfully. I will intersect the resulting polygons with the city's (multi)polygon.

I don't think I really understand the reason leaflet doesn't work with geoVoronoi. Are the resulting coordinates imprecise? I tried reading the README but could not crack how it should work either. Is it even possible to convert the results of geoVoronoi into a planar map like Leaflet? I thought that when I get GPS coordinates as a result, they would just work on the map...

Also, do you then agree that it would be better to use d3-delaunay instead of d3-geo-voronoi for @turf/voronoi? Or is there a way to make it work here?

@Fil
Copy link

Fil commented Jan 2, 2024

I thought that when I get GPS coordinates as a result, they would just work on the map...

They do! The points are at the correct location. What is different is line interpolation (the actual sequence of intermediary points that describes a line on the screen). Leaflet draws straight lines between points, meaning it expects the user to have done line interpolation in advance if they have complex paths, whereas D3 does line interpolation during projection.

Is it even possible to convert the results of geoVoronoi into a planar map like Leaflet?

You can convert spherical coordinates to equirectangular coordinates using d3.geoProject (from the d3-geo-projection module):

const geoVoronoi = d3.geoVoronoi()(points).polygons();
const voronoi = d3.geoProject(geoVoronoi, d3.geoEquirectangular().precision(0.1).reflectY(true).translate([0,0]).scale(180/Math.PI));

In terms of point projection, this is a no-op: a point like [longitude, latitude] is projected to the same values. However, this code computes the proper line interpolations, which you can then pass to leaflet.

But on top of that problem, there is the additional issue of instability (Fil/d3-geo-voronoi#10). You can solve this currently, for this dataset, by adding a small perturbation to the coordinates:

 return [point.lng+0.01*(Math.random()-.5), point.lat+0.01*(Math.random()-.5)]

but of course this is not fully satisfying.

do you then agree that it would be better to use d3-delaunay instead of d3-geo-voronoi for @turf/voronoi

I don't know. For local or regional data it seems more straightforward (and faster) to use d3-delaunay, but the larger the area, the more incorrect it gets, so for global data you need the spherical approach.

@maral
Copy link
Contributor

maral commented Jan 2, 2024

Thanks for a thorough and clear explanation.

Well, that sucks. 😂 I really wanted to have a one-size-fits-all solution, which is not the case here, unfortunately. For my use case, I will use the planar version, for @turf/voronoi, I am not really sure what is the best approach then. Maybe even having two algorithms the user can choose from might be an option...

For now, the algorithm skews the polygons (since it works with cartesian coordinates instead of WGS-84) and even places the polygons outside the points, so basically anything would be better than the current version. What do you think?

@Fil
Copy link

Fil commented Jan 2, 2024

I'm not sure why polygons would be "outside the points", an explicit example would help to understand.

In my understanding, skewing happens because the local plane expressed in degrees is not isotropic: a distance of 10 km to the East or West doesn't correspond to the same number of degrees as the same distance to the North or South (unless you are near the equator). (In other words, a circle is mapped to an ellipse.) This is not tied to d3.voronoi, it will be the same when you replace d3.voronoi by d3.delaunay.

Your approach "convert the WGS-84 points to mercator projection, apply planar voronoi and then convert the coordinates back to WGS-84" matches the Leaflet projection, so it sounds to me like the best approach if you want to make polygons "look" correct.

For local data, the approximation will be very good, even though not optimal (using a projection tangent to the local plane would be even better… but it would involve a determination of the local plane).

This amounts to what we do in Observable Plot with the voronoi mark, where we compute it on the projected points; except here, you need to go back to WGS-84 because you're passing the polygons to Leaflet, which will then project them (again) with Mercator 😀.

I'd still recommend to move from d3.voronoi to d3.delaunay, because performance and precision are orders of magnitude better.

A problem you will have is if you want to draw a voronoi of points that are close to the antemeridian. In that case you'll need to add a rotation in your intermediate mercator.

If we could fix Fil/d3-geo-voronoi#10 I'd recommend the more complex but exact approach I outlined above (geoVoronoi + geoProjection to WSG-84 for line interpolation).

@maral
Copy link
Contributor

maral commented Jan 2, 2024

I'm not sure why polygons would be "outside the points", an explicit example would help to understand.

I meant, when I tried applying plain @turf/voronoi to the same data set as above, some points were outside their voronoi cell, i.e. in a neighboring cell that then contained 2 points at the same time. I'm not sure I have the energy to put together an example though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants