Visualiser¶
In this notebook we'll explore how to finally visualise the result of an entire workflow (examples [1-5]) into a maps—static and interactive visualisers.
Let’s make an enriched layer pop!
Data source used:
- PLUTO data from NYC Open Data. https://www.nyc.gov/content/planning/pages/resources/datasets/mappluto-pluto-change
Let’s make some pretty maps! 🗺️
In [1]:
Copied!
import urban_mapper as um
# Kick off UrbanMapper
mapper = um.UrbanMapper()
import urban_mapper as um
# Kick off UrbanMapper
mapper = um.UrbanMapper()
--------------------------------------------------------------------------- ModuleNotFoundError Traceback (most recent call last) Cell In[1], line 1 ----> 1 import urban_mapper as um 3 # Kick off UrbanMapper 4 mapper = um.UrbanMapper() File ~/checkouts/readthedocs.org/user_builds/urbanmapper/checkouts/70/src/urban_mapper/__init__.py:3 1 from loguru import logger ----> 3 from .mixins import ( 4 LoaderMixin, 5 EnricherMixin, 6 VisualMixin, 7 TableVisMixin, 8 AuctusSearchMixin, 9 PipelineGeneratorMixin, 10 UrbanPipelineMixin, 11 ) 12 from .modules import ( 13 LoaderBase, 14 CSVLoader, (...) 30 PipelineGeneratorFactory, 31 ) 33 from .urban_mapper import UrbanMapper File ~/checkouts/readthedocs.org/user_builds/urbanmapper/checkouts/70/src/urban_mapper/mixins/__init__.py:1 ----> 1 from .loader import LoaderMixin 2 from .enricher import EnricherMixin 3 from .visual import VisualMixin File ~/checkouts/readthedocs.org/user_builds/urbanmapper/checkouts/70/src/urban_mapper/mixins/loader.py:1 ----> 1 from urban_mapper.modules.loader.loader_factory import LoaderFactory 4 class LoaderMixin(LoaderFactory): 5 def __init__(self): File ~/checkouts/readthedocs.org/user_builds/urbanmapper/checkouts/70/src/urban_mapper/modules/__init__.py:1 ----> 1 from .loader import LoaderBase, CSVLoader, ShapefileLoader, ParquetLoader 2 from .imputer import ( 3 GeoImputerBase, 4 SimpleGeoImputer, 5 AddressGeoImputer, 6 ) 7 from .filter import ( 8 GeoFilterBase, 9 BoundingBoxFilter, 10 ) File ~/checkouts/readthedocs.org/user_builds/urbanmapper/checkouts/70/src/urban_mapper/modules/loader/__init__.py:3 1 from .abc_loader import LoaderBase 2 from .loaders import CSVLoader, ShapefileLoader, ParquetLoader ----> 3 from .loader_factory import LoaderFactory 5 __all__ = [ 6 "LoaderBase", 7 "CSVLoader", (...) 10 "LoaderFactory", 11 ] File ~/checkouts/readthedocs.org/user_builds/urbanmapper/checkouts/70/src/urban_mapper/modules/loader/loader_factory.py:19 17 from urban_mapper.modules.loader.loaders.csv_loader import CSVLoader 18 from urban_mapper.modules.loader.loaders.parquet_loader import ParquetLoader ---> 19 from urban_mapper.modules.loader.loaders.raster_loader import RasterLoader # Importing RasterLoader of the new raster loader module 20 from urban_mapper.modules.loader.loaders.shapefile_loader import ShapefileLoader 21 from urban_mapper.utils import require_attributes File ~/checkouts/readthedocs.org/user_builds/urbanmapper/checkouts/70/src/urban_mapper/modules/loader/loaders/raster_loader.py:2 1 from ..abc_loader import LoaderBase ----> 2 import rasterio 3 from typing import Any 4 import numpy as np ModuleNotFoundError: No module named 'rasterio'
Preparing the Data and Layer¶
First, let’s load data, create a layer, and enrich it with average building floors.
Note that:
- Loader example can be seen in
examples/Basics/loader.ipynb - Urban Layer example can be seen in
examples/Basics/urban_layer.ipynb - Enricher example can be see in
examples/Basics/enricher.ipynb - Imputer example can be seen in
examples/Basics/imputer.ipynb - Filter example can be seen in
examples/Basics/filter.ipynb
In [2]:
Copied!
######
#
# We will on purpose do inline chaining of the methods to fasten the process.
#
######
# Load data
# Note: For the documentation interactive mode, we only query 5000 records from the dataset. Feel free to remove for a more realistic analysis.
data = mapper.loader.from_huggingface("oscur/pluto", number_of_rows=5000, streaming=True).with_columns("longitude", "latitude").load()
# Create urban layer
layer = mapper.urban_layer.with_type("streets_intersections").from_place("Downtown Brooklyn, New York City, USA").build()
# Impute your data if they contain missing values
data = mapper.imputer.with_type("SimpleGeoImputer").on_columns("longitude", "latitude").transform(data, layer)
# Filter your data if they contain data beyond the bounding box your urban layer above
data = mapper.filter.with_type("BoundingBoxFilter").transform(data, layer)
# Map data to nearest layer
_, mapped_data = layer.map_nearest_layer(
data,
longitude_column="longitude",
latitude_column="latitude",
output_column="nearest_intersection",
)
# Enrich layer
enricher = (
mapper.enricher
.with_data(group_by="nearest_intersection", values_from="numfloors")
.aggregate_by(method="mean", output_column="avg_floors")
.build()
)
enriched_layer = enricher.enrich(mapped_data, layer)
######
#
# We will on purpose do inline chaining of the methods to fasten the process.
#
######
# Load data
# Note: For the documentation interactive mode, we only query 5000 records from the dataset. Feel free to remove for a more realistic analysis.
data = mapper.loader.from_huggingface("oscur/pluto", number_of_rows=5000, streaming=True).with_columns("longitude", "latitude").load()
# Create urban layer
layer = mapper.urban_layer.with_type("streets_intersections").from_place("Downtown Brooklyn, New York City, USA").build()
# Impute your data if they contain missing values
data = mapper.imputer.with_type("SimpleGeoImputer").on_columns("longitude", "latitude").transform(data, layer)
# Filter your data if they contain data beyond the bounding box your urban layer above
data = mapper.filter.with_type("BoundingBoxFilter").transform(data, layer)
# Map data to nearest layer
_, mapped_data = layer.map_nearest_layer(
data,
longitude_column="longitude",
latitude_column="latitude",
output_column="nearest_intersection",
)
# Enrich layer
enricher = (
mapper.enricher
.with_data(group_by="nearest_intersection", values_from="numfloors")
.aggregate_by(method="mean", output_column="avg_floors")
.build()
)
enriched_layer = enricher.enrich(mapped_data, layer)
--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[2], line 9 1 ###### 2 # 3 # We will on purpose do inline chaining of the methods to fasten the process. (...) 7 # Load data 8 # Note: For the documentation interactive mode, we only query 5000 records from the dataset. Feel free to remove for a more realistic analysis. ----> 9 data = mapper.loader.from_huggingface("oscur/pluto", number_of_rows=5000, streaming=True).with_columns("longitude", "latitude").load() 11 # Create urban layer 12 layer = mapper.urban_layer.with_type("streets_intersections").from_place("Downtown Brooklyn, New York City, USA").build() NameError: name 'mapper' is not defined
Static Visualisation¶
Now that we've cooked the various ingredients, let’s whip up a static map with Matplotlib to see our enriched layer.
In [3]:
Copied!
# Static visualisation
fig_static = (
mapper
.visual # From the visualiser module
.with_type("Static") # With type Static
.show(columns=["avg_floors"]) # Show the avg_floors column
.render(enriched_layer.get_layer()) # Render the enriched layer
)
# Static visualisation
fig_static = (
mapper
.visual # From the visualiser module
.with_type("Static") # With type Static
.show(columns=["avg_floors"]) # Show the avg_floors column
.render(enriched_layer.get_layer()) # Render the enriched layer
)
--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[3], line 3 1 # Static visualisation 2 fig_static = ( ----> 3 mapper 4 .visual # From the visualiser module 5 .with_type("Static") # With type Static 6 .show(columns=["avg_floors"]) # Show the avg_floors column 7 .render(enriched_layer.get_layer()) # Render the enriched layer 8 ) NameError: name 'mapper' is not defined
Interactive Visualisation¶
Now, let’s go fancy with an interactive Folium map—dark theme, because why not?
In [4]:
Copied!
# Interactive visualisation
fig_interactive = (
mapper
.visual # From the visualiser module
.with_type("Interactive") # With type Interactive
.with_style({"tiles": "CartoDB dark_matter", "colorbar_text_color": "white"}) # Dark theme style
.show(columns=["avg_floors"]) # Show the avg_floors column
.render(enriched_layer.get_layer()) # Render the enriched layer
)
fig_interactive
# Interactive visualisation
fig_interactive = (
mapper
.visual # From the visualiser module
.with_type("Interactive") # With type Interactive
.with_style({"tiles": "CartoDB dark_matter", "colorbar_text_color": "white"}) # Dark theme style
.show(columns=["avg_floors"]) # Show the avg_floors column
.render(enriched_layer.get_layer()) # Render the enriched layer
)
fig_interactive
--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[4], line 3 1 # Interactive visualisation 2 fig_interactive = ( ----> 3 mapper 4 .visual # From the visualiser module 5 .with_type("Interactive") # With type Interactive 6 .with_style({"tiles": "CartoDB dark_matter", "colorbar_text_color": "white"}) # Dark theme style 7 .show(columns=["avg_floors"]) # Show the avg_floors column 8 .render(enriched_layer.get_layer()) # Render the enriched layer 9 ) 10 fig_interactive NameError: name 'mapper' is not defined
Be Able To Preview Your Visualiser¶
Want to check your visualiser’s setup? preview() shows you the type and style—perfect for shared projects!
In [5]:
Copied!
# Preview visualiser
print(mapper.visual.preview())
# Preview visualiser
print(mapper.visual.preview())
--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[5], line 2 1 # Preview visualiser ----> 2 print(mapper.visual.preview()) NameError: name 'mapper' is not defined
Provide many different datasets to the same static visualization¶
You can load many datasets and feed the enricher with a dictionary. All the provided datasets should have the same columns provided in with_data, aggregate_by, etc.
The static visualizer looks into the enriched data with data_id column and uses it to show data with different markers.
In [6]:
Copied!
# Load CSV data
data1 = (
mapper
.loader
.from_huggingface("oscur/pluto", number_of_rows=1000, streaming=True).with_columns("longitude", "latitude").load()
# From the loader module, from the following file and with the `longitude` and `latitude`
)
# Load Parquet data
data2 = (
mapper
.loader
.from_huggingface("oscur/taxisvis1M", number_of_rows=1000, streaming=True) # To update with your own path
.with_columns("pickup_longitude", "pickup_latitude") # Inform your long and lat columns
.with_map({"pickup_longitude": "longitude", "pickup_latitude": "latitude"}) ## Routines like layer.map_nearest_layer needs datasets with the same longitude_column and latitude_column
.load()
)
data = {
"pluto_data": data1,
"taxi_data": data2,
}
# Create a new urban layer to the data
layer = mapper.urban_layer.with_type("streets_intersections").from_place("Downtown Brooklyn, New York City, USA").build()
# Map datasets to the nearest layer
# Here the point is to say which intersection of the city maps with which record(s) in each of your datasets
# so that we can take into account when enriching.
_, mapped_data = layer.map_nearest_layer(
data,
longitude_column="longitude",
latitude_column="latitude",
output_column="nearest_intersection", # Will create this column in the data, so that we can re-use that throughout the enriching process below.
)
# Set up and apply enricher with debug enabled
enricher = (
mapper
.enricher # From the enricher module
.with_data(
group_by="nearest_intersection", values_from="numfloors", data_id="pluto_data"
) # Reading: With data grouped by the nearest intersection, and the values from the attribute numfloors
#Both datasets should have the same group_by and values_from columns
.aggregate_by(
method="mean", output_column="avg_floors"
) # Reading: Aggregate by using the mean and output the computation into the avg_floors new attribute of the urban layer
.with_debug() # Enable debug to add DEBUG_avg_floors column which will contain the list of indices from the input data used for each enrichment
.build()
)
enriched_layer = enricher.enrich(
mapped_data, layer
) # Data to use, Urban Layer to Enrich.
# Static visualisation
fig_static = (
mapper
.visual # From the visualiser module
.with_type("Static") # With type Static
.show(columns=["avg_floors"]) # Show the avg_floors column
.render(enriched_layer.get_layer()) # Render the enriched layer
)
# Load CSV data
data1 = (
mapper
.loader
.from_huggingface("oscur/pluto", number_of_rows=1000, streaming=True).with_columns("longitude", "latitude").load()
# From the loader module, from the following file and with the `longitude` and `latitude`
)
# Load Parquet data
data2 = (
mapper
.loader
.from_huggingface("oscur/taxisvis1M", number_of_rows=1000, streaming=True) # To update with your own path
.with_columns("pickup_longitude", "pickup_latitude") # Inform your long and lat columns
.with_map({"pickup_longitude": "longitude", "pickup_latitude": "latitude"}) ## Routines like layer.map_nearest_layer needs datasets with the same longitude_column and latitude_column
.load()
)
data = {
"pluto_data": data1,
"taxi_data": data2,
}
# Create a new urban layer to the data
layer = mapper.urban_layer.with_type("streets_intersections").from_place("Downtown Brooklyn, New York City, USA").build()
# Map datasets to the nearest layer
# Here the point is to say which intersection of the city maps with which record(s) in each of your datasets
# so that we can take into account when enriching.
_, mapped_data = layer.map_nearest_layer(
data,
longitude_column="longitude",
latitude_column="latitude",
output_column="nearest_intersection", # Will create this column in the data, so that we can re-use that throughout the enriching process below.
)
# Set up and apply enricher with debug enabled
enricher = (
mapper
.enricher # From the enricher module
.with_data(
group_by="nearest_intersection", values_from="numfloors", data_id="pluto_data"
) # Reading: With data grouped by the nearest intersection, and the values from the attribute numfloors
#Both datasets should have the same group_by and values_from columns
.aggregate_by(
method="mean", output_column="avg_floors"
) # Reading: Aggregate by using the mean and output the computation into the avg_floors new attribute of the urban layer
.with_debug() # Enable debug to add DEBUG_avg_floors column which will contain the list of indices from the input data used for each enrichment
.build()
)
enriched_layer = enricher.enrich(
mapped_data, layer
) # Data to use, Urban Layer to Enrich.
# Static visualisation
fig_static = (
mapper
.visual # From the visualiser module
.with_type("Static") # With type Static
.show(columns=["avg_floors"]) # Show the avg_floors column
.render(enriched_layer.get_layer()) # Render the enriched layer
)
--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[6], line 3 1 # Load CSV data 2 data1 = ( ----> 3 mapper 4 .loader 5 .from_huggingface("oscur/pluto", number_of_rows=1000, streaming=True).with_columns("longitude", "latitude").load() 6 # From the loader module, from the following file and with the `longitude` and `latitude` 7 ) 9 # Load Parquet data 10 data2 = ( 11 mapper 12 .loader (...) 16 .load() 17 ) NameError: name 'mapper' is not defined
More visualiser primitives ?¶
Wants more? Come shout that out on https://github.com/VIDA-NYU/UrbanMapper/issues/9
Wrapping Up¶
Blimey, you’re ace! 🌟 You’ve made static and interactive maps like a pro. Tweak the styles or columns next—you’re unstoppable! 🚀