Getting Pydeck to Play Nicely with GeoPandas.

How To

Geospatial

pydeck

geopandas

Building Pydeck Maps from GeoPandas GeoDataFrames.

Author

Rich Leyshon

Published

February 18, 2024

Introduction

Pydeck is a python client for pydeck.gl, a powerful geospatial visualisation library. It’s a relatively new library and integrating it with the existing python geospatial ecosystem is currently a little tricky. This article demonstrates how to build pydeck ScatterplotLayer and GeoJsonLayer from geopandas GeoDataFrames.

A Note on the Purpose

The content of this article was written using pydeck 0.8.0. Future releases may alter the package behaviour.

Intended Audience

Python practitioners familiar with virtual environments, requests and geospatial analysis with geopandas.

The Scenario

You have a geopandas GeoDataFrame with point or polygon geometries. You are attempting to build a pydeck visualisation but end up with empty basemap tiles.

What You’ll Need:

Preferred python environment manager (eg conda)
Python package manager (eg pip)
requirements.txt:

requirements.txt

geopandas
pandas
pydeck
requests

Prepare Environment

Create a virtual environment.
Install the required dependencies.
Activate the virtual environment.
Create a python script and import the dependencies.

import json

import geopandas as gpd
import numpy as np
import pandas as pd
import pydeck as pdk
import requests
from sklearn import preprocessing

Build a `ScatterplotLayer`

Ingest Data

For the point data, I will ingest all Welsh lower super output area 2021 population-weighted centroids from ONS Open Geography Portal.

For more on working with ONS Open Geography Portal, see Getting Data from ONS Open Geography Portal.

Show the code

ENDPOINT = "https://services1.arcgis.com/ESMARspQHYMw9BZ9/arcgis/rest/services/LLSOA_Dec_2021_PWC_for_England_and_Wales_2022/FeatureServer/0/query"
PARAMS = {
    "where": "LSOA21CD like 'W%'",
    "f": "geoJSON", 
    "outFields": "*",
    "outSR": 4326,
}
resp = requests.get(ENDPOINT, params=PARAMS)
if resp.ok:
    content = resp.json()
else:
    raise requests.RequestException(f"HTTP {resp.status_code} : {resp.reason}")

centroids = gpd.GeoDataFrame.from_features(
    features=content["features"], crs=content["crs"]["properties"]["name"])
centroids.head()

	geometry	FID	LSOA21CD	GlobalID
0	POINT (-3.27237 52.76593)	68	W01000461	205abce3-aa73-408a-ab25-1c8364c12a63
1	POINT (-3.21395 51.77614)	92	W01001459	963a18d4-f225-4273-80aa-26c01236af11
2	POINT (-3.20681 51.78261)	133	W01001456	71e6c3f6-7e87-4cea-9256-97aa366cac43
3	POINT (-2.68552 51.64398)	194	W01001585	df9d1950-64cd-42d4-a8fd-18c989951e27
4	POINT (-3.09114 51.82767)	203	W01001563	b2cc735d-c637-40dc-9871-cb8ac5bca9c3

The geometry column is not in a format that pydeck will accept. Adding a column with a list of long,lat values for each coordinate will do the trick.

centroids["pydeck_geometry"] = [[c.x, c.y] for c in centroids["geometry"]]
centroids.head()

	geometry	FID	LSOA21CD	GlobalID	pydeck_geometry
0	POINT (-3.27237 52.76593)	68	W01000461	205abce3-aa73-408a-ab25-1c8364c12a63	[-3.27236528249052, 52.7659297296924]
1	POINT (-3.21395 51.77614)	92	W01001459	963a18d4-f225-4273-80aa-26c01236af11	[-3.2139466137404, 51.7761398048545]
2	POINT (-3.20681 51.78261)	133	W01001456	71e6c3f6-7e87-4cea-9256-97aa366cac43	[-3.20681397850058, 51.7826095824385]
3	POINT (-2.68552 51.64398)	194	W01001585	df9d1950-64cd-42d4-a8fd-18c989951e27	[-2.68552245106253, 51.6439814411607]
4	POINT (-3.09114 51.82767)	203	W01001563	b2cc735d-c637-40dc-9871-cb8ac5bca9c3	[-3.09114462316771, 51.8276689661644]

Pydeck Visualisation

With the correct geometry format, the scatterplot is trivial.

Tip

Control the map by click and dragging the map with your mouse. Hold shift + click and drag to yaw or pitch the map. Scroll in and out to alter the zoom.

scatter = pdk.Layer(
    "ScatterplotLayer",
    centroids,
    pickable=True,
    stroked=True,
    filled=True,
    line_width_min_pixels=1,
    get_position="pydeck_geometry",
    get_fill_color=[255, 140, 0],
    get_line_color=[255, 140, 0],
    radius_min_pixels=3,
    opacity=0.1,
)
# Set the viewport location
view_state = pdk.ViewState(
    longitude=-2.84,
    latitude=52.42,
    zoom=6.5,
    max_zoom=15,
    pitch=0,
    bearing=0,
)
tooltip = {
    "text": "LSOA21CD: {LSOA21CD}"
}
# Render
r = pdk.Deck(
    layers=scatter, initial_view_state=view_state, tooltip=tooltip
)
r

Build a `GeoJsonLayer`

GeoJsonLayer is what tends to be used for presenting polygons with pydeck maps. The pydeck docs GeoJsonLayer example uses geoJSON data hosted on GitHub. But with a little effort, a Geopandas GeoDataFrame can be coerced to the necessary format.

Ingest Data

To demonstrate working with polygons, the Welsh super generalised 2023 local authority district boundaries will be ingested from ONS Open Geography Portal.

As elevation and polygon colour will be controlled by features of the data, sklearn.prepeocessing is used to scale the “Shape__Area” column.

Show the code

ENDPOINT="https://services1.arcgis.com/ESMARspQHYMw9BZ9/arcgis/rest/services/Local_Authority_Districts_December_2023_Boundaries_UK_BSC/FeatureServer/0/query"
PARAMS["where"] = "LAD23CD like 'W%'"
resp = requests.get(ENDPOINT, params=PARAMS)
if resp.ok:
    content = resp.json()
else:
    raise requests.RequestException(f"HTTP {resp.status_code} : {resp.reason}")

polygons = gpd.GeoDataFrame.from_features(
    features=content["features"], crs=content["crs"]["properties"]["name"])
# feature engineering for pydeck viz
min_max_scaler = preprocessing.MinMaxScaler()
x = polygons["Shape__Area"].values.reshape(-1, 1)
x_scaled = min_max_scaler.fit_transform(x)
polygons["area_norm"] = pd.Series(x_scaled.flatten())
polygons.head()

	geometry	FID	LAD23CD	LAD23NM	LAD23NMW	BNG_E	BNG_N	LONG	LAT	Shape__Area	Shape__Length	GlobalID	area_norm
0	MULTIPOLYGON (((-4.02145 53.32145, -4.03186 53...	340	W06000001	Isle of Anglesey	Ynys Môn	245217	378331	-4.32298	53.27931	7.137720e+08	226738.375131	dd2ba2ec-78e0-4113-bfec-5bbf7828fb0b	0.118905
1	MULTIPOLYGON (((-3.93116 52.55401, -3.93293 52...	341	W06000002	Gwynedd	Gwynedd	280555	334966	-3.77715	52.89883	2.550513e+09	466258.144553	9436112f-3572-409b-8d93-cf21aaca60d5	0.479954
2	POLYGON ((-3.86356 53.34172, -3.84075 53.33698...	342	W06000003	Conwy	Conwy	283293	362563	-3.74646	53.14739	1.131701e+09	220424.522936	d9ae4b38-3437-4e10-8556-03e3b529c5c5	0.201058
3	POLYGON ((-3.37625 53.32772, -3.38835 53.32323...	343	W06000004	Denbighshire	Sir Ddinbych	309843	355416	-3.34761	53.08833	8.374441e+08	200171.915162	22313345-a2d2-4a54-9edc-1f492cd7320e	0.143215
4	MULTIPOLYGON (((-3.08242 53.25551, -3.08806 53...	344	W06000005	Flintshire	Sir y Fflint	321134	369280	-3.18248	53.21471	4.386527e+08	146522.477076	693217ec-5c94-4ec1-9d81-eed3f99f1777	0.064825

In order to pass the content of this GeoDataFrame to pydeck, use the to_json method to format as a geoJSON string. Then use json.loads() to format that string as a dictionary.

# format data for use in pydeck
json_out = json.loads(polygons.to_json())
# inspect the first authority
json_out["features"][0]["properties"]

{'FID': 340,
 'LAD23CD': 'W06000001',
 'LAD23NM': 'Isle of Anglesey',
 'LAD23NMW': 'Ynys Môn',
 'BNG_E': 245217,
 'BNG_N': 378331,
 'LONG': -4.32298,
 'LAT': 53.27931,
 'Shape__Area': 713772027.9524,
 'Shape__Length': 226738.375130659,
 'GlobalID': 'dd2ba2ec-78e0-4113-bfec-5bbf7828fb0b',
 'area_norm': 0.11890511689989425}

Pydeck Visualisation

This format can now be passed to pydeck. One ‘gotcha’ to be aware of, when using attributes in the json to control elevation or colour, the json properties must be explicitly referenced, eg "properties.area_norm".

In contrast, when using json attributes in the tooltip, you can refer to them directly, eg "area_norm".

r = "100"
g = "(1 - properties.area_norm) * 255"
b = "properties.area_norm * 255"
fill = f"[{r},{g},{b}]"
geojson = pdk.Layer(
        "GeoJsonLayer",
        json_out,
        pickable=True,
        opacity=1,
        stroked=True,
        filled=True,
        extruded=True,
        wireframe=True,
        auto_highlight=True,
        get_elevation="properties.area_norm * 200",
        elevation_scale=100,
        get_fill_color=fill,
    )
tooltip = {"text": "{LAD23NM}\n{LAD23CD}"}
view_state = pdk.ViewState(
    longitude=-2.84,
    latitude=52.42,
    zoom=6.5,
    max_zoom=15,
    pitch=100,
    bearing=33,
)
r = pdk.Deck(
    layers=geojson,
    initial_view_state=view_state,
    tooltip=tooltip,
)
r

Tips

pydeck does not raise when layer data are not formatted correctly. This can result in some lengthy render times only to discover you have an empty map. To combat this, work with the head or some small sample of your data until you have your map working.

Conclusion

This article has recorded the state of play between pydeck and geopandas at the time of writing. Specifically, formatting:

geometry columns for pydeck ScatterplotLayer
a GeoDataFrame for use with pydeck GeoJsonLayer.

I hope it saves someone some time bashing geopandas about.

fin!

Introduction

Intended Audience

The Scenario

What You’ll Need:

Prepare Environment

Build a ScatterplotLayer

Ingest Data

Pydeck Visualisation

Build a GeoJsonLayer

Ingest Data

Pydeck Visualisation

Tips

Conclusion

Build a `ScatterplotLayer`

Build a `GeoJsonLayer`