geowright: testing a map you cannot see into
geowright is a grey-box testing layer for canvas maps in Playwright, with an adapter for MapLibre GL JS and Mapbox GL JS. It lets a test assert that a feature is on the map and act on one: click it, right-click a location, drag a vertex, draw a polygon. It is mine, it is Apache-2.0, and it is not published yet.
The interesting number on this page is not a feature count. I ran the library against 33 map applications written by people who have never heard of it, and 16 of them needed a change to the library before their tests could pass. My own demo app had been hiding all of it.
The first version took one day of heavily AI-assisted work, and that speed is why I tested it against apps I did not write: moving that fast is exactly what leaves an author agreeing with his own assumptions.
- 16 of 33outside apps needed a library fix
- 349tests in the main suite
What a DOM-only suite cannot do
A WebGL map is one <canvas> element. To Playwright, and to an accessibility tree, and to a browser agent, everything drawn inside it is a single opaque rectangle. There is no element for the alert pin, no role, no accessible name, nothing to select. So a normal end-to-end suite can drive the panel beside the map and then has to stop at the only question that matters, which is whether the thing the user came for is actually on the map.
That leaves two gaps, and the library exists to close them. The first is assertion: after an action somewhere in the normal interface, verify that something was added to, changed on, or removed from the map. The second is action: do the things that start on the map itself. Both need the test to see inside the canvas, which is what makes this grey-box rather than black-box. The application registers its map instance on one line and the adapter is injected by Playwright at test time, so nothing is imported and nothing ships to production.
The line I held throughout is that actions are real input. Every click is page.mouse at a projected pixel, never a synthetic event fired at the map object. A test that calls the library's own event emitter proves the emitter works. It does not prove a user can click the pin, which is the entire question.
The app side is the one line; a test reads like any other Playwright test:
const map = new maplibregl.Map({ ... });
// The whole hook: nothing is imported,
// nothing ships.
(window.__geowright_maps ??= {}).main = map;import { geowright, expect } from 'geowright';
const map = geowright(page);
await map.waitForIdle();
await expect(map).toHaveFeature({
layer: 'alerts',
where: { severity: 'high' },
});
// an action in the app's normal UI...
await page.getByTestId('alert-remove-2').click();
// ...and its effect on the map
await expect(map).not.toHaveFeature({
layer: 'alerts',
id: 2,
});
await map
.feature({ layer: 'alerts', id: 4 })
.click();The apps I did not write
Here is the problem with a library like this. The demo application I tested against was written by me, on the same day, with the library's assumptions already in my head. Every map in it had feature ids, because I gave them ids. Every layer had a name I chose. A suite that passes against that proves the author agrees with himself.
So I vendored 33 applications written by other people, under their own licenses: MapLibre's own published examples, and real applications built on the React, Svelte and Vue wrappers. Each one gets the one-line hook and nothing else. The rule is enforced mechanically rather than by intention: each vendored example carries a diff against the upstream file, a script re-fetches upstream and checks it, and the diff has to be exactly one added line. If testing an application requires editing it, the library is what has to change.
17 of the 33 worked against the library as it stood. 16 did not, and every one of those became a library change rather than a workaround in the test. That is the measure I actually wanted: not how many applications it supports, but how much of its apparent success had been my own demo agreeing with me.
| What I had assumed | What a real app did | What changed |
|---|---|---|
| Features have ids | Three points of interest, no ids, identical properties. All three counted as one feature. | Identity became id plus properties plus where the feature is drawn, so identical features are told apart by position. |
| Layer ids are meaningful | A wrapper generates them. A Svelte app's layer is called svmlgl-layer-0, which no test author would write. | Query by what the layer is rather than what it is called: layer type, source, or both. |
| The map is on screen | A Svelte clusters app puts the map below the fold, so every pointer action aimed at nothing. | Scroll the map into the viewport before an action, as step zero of actionability. |
| A feature sits at a pixel | A 3D building is drawn as a lifted roof and walls, far from its footprint on the ground. | Act where the feature is drawn: roof first, then walls, then footprint. |
| The library's own query is reliable | On a globe projection, the map library's viewport-wide query returns a fraction of the visible features, some on the far side, and nothing at all for 3D extrusions. | A workaround built from public API only: a grid of box queries filtered to the near side, plus point queries for extrusions. The one case it can still miss is documented rather than hidden. |
| Duplicate world copies are harmless | A Vue filter app rendered the same features in repeated world copies, which the new identity rule counted twice. | Positions are folded by the width the world repeats at, so the copies merge back into one feature. The next application found it, a regression of my own fix from the same day, and a spec pins it. |
The caveats belong here too, because the number flatters me otherwise. The applications were chosen at selection time for permissive licensing, so nothing was tried and abandoned, and one was excluded because a video in it had no license. The fixes are all mine, made the same day, by the one person who could read both sides of the problem. An outside team that hit three points of interest counting as one would have filed a bug or walked away.
Rendered is not the same as visible
The sharpest correctness trap in this domain is one the map library hands you. Its query for rendered features also returns features painted at zero opacity, in a transparent color, at zero radius, or with their text hidden. So the obvious assertion, that a feature is on the map, passes happily on a feature no user can see. A suite built on it would go green on a map that renders nothing.
Queries match only what a user can see, by default. The layer's paint properties are evaluated for that specific feature, through the map library's own expression evaluator, so data-driven styles, zoom and feature state are all accounted for. A failure says which property made it invisible rather than only that nothing matched. There is an opt-out for the cases where merely rendered is what you mean, and a stricter check on top that probes the canvas pixels directly, toggling the layer and comparing, which catches an icon whose image is transparent.
The same principle runs through the failure output, which is the part I would keep if I had to throw the rest away. When an assertion fails it prints what was actually on the map: the camera, every feature in the layer with its properties and its pixel, and the values the property you filtered on really had. It distinguishes a layer that does not exist from a hidden layer, from a feature that is off screen, from one occluded by another feature, from one covered by a popup, and it names the occluder. A failing test that says only that it expected one and found zero has made you do the work twice. This is the failure the library prints when a test asks for a severity no alert has:
expect(map).toHaveFeature({"layer":"alerts","where":{"severity":"critical"}})
Expected: at least one rendered feature matching, within 7000ms
Received: 0 match(es)
What is on the map:
view: {"center":{"lng":34.78,"lat":32.08},"zoom":12, ...}
layer "alerts": 3 feature(s) rendered in viewport
- alerts#3 {"id":3,"name":"Noise complaint","severity":"low"} @(363.5,428.7)
- alerts#2 {"id":2,"name":"Road closure","severity":"medium"} @(538.3,256.9)
- alerts#1 {"id":1,"name":"Gas leak","severity":"high"} @(421.8,325.6)
"severity" wanted "critical"; rendered values: "low", "medium", "high"Real input, verified at the click
An action waits for a sequence of conditions before it fires, and each one is there because skipping it produces a test that lies:
- The query resolves to exactly one feature, and the style actually paints it.
- The map is still, with the target pixel stable across two frames.
- The pixel is on the feature. For a polygon that means an interior point well away from the edge rather than the centroid, because the centroid of a horseshoe is outside it.
- The feature is the topmost hit at that pixel, discounting its own label and anything painted invisibly.
- No popup or control covers it.
- At the instant the real pointer event arrives, the hit test runs again. If the target is no longer there, the press is swallowed before the application sees it and the action retries.
The last condition is the guard that makes the rest safe, and it is what lets the hardest case work at all. A feature that never stops moving, an animated vehicle, fails the stability check forever, so the pointer is instead aimed ahead of the target along its measured velocity, and how far ahead is learned from each miss. A press that arrives late is swallowed rather than delivered somewhere wrong. It can take several attempts and it can time out. It does not click the wrong thing, and the difference between those two failure modes is the whole point.
Work this fiddly needs a flake bar rather than a green run: the whole suite runs twenty times over (--repeat-each=20) with no retries configured anywhere. The first time, it took five attempts to get there. The first two found three genuine flakes, all in code written that day, all fixed. The third was two failures deep inside a load spike from unrelated work on the same machine, and the fourth was ruined the same way. Only the fifth had the machine to itself. The suite as it stood on 24 September, 349 tests, passed 6,980 runs out of 6,980.
Where it actually stands
349 tests in the main suite and 60 more in a version matrix that runs the same core specs against Mapbox GL JS 1.13 and MapLibre 2 through 6, as its own job, so a tick in the support table means asserted rather than assumed. Newer methods are feature-detected instead of version-sniffed, and what an old version cannot do is said in the failure message rather than answered wrongly. Continuous integration runs the suite, the matrix and the oldest supported Playwright as three jobs, and the suite has passed on Linux in Playwright's own container.
It is early. It is not on npm, it has one adapter, and the public repository does not exist yet, so there is nothing to link. The open questions are written down in the design log rather than left to be discovered: touch gestures need a browser-specific protocol and so run on Chromium only, hovering a moving target is weaker than clicking one and is documented as weak rather than quietly shipped, and an extrusion narrower than sixteen pixels on a globe can still be missed by a count.
The origin is older than the repository. In an earlier role I inherited a test suite where map interaction had been filed as out of scope, because the framework the team used could not do it, and the thing that changed the answer was an undocumented internal library I had to read until I understood it. That story is on the earlier work page. geowright is the version of that capability that does not depend on finding someone else's undocumented package on a network share.