Three Pipelines Stacked in JGIS¶
Collisions, Taxi Trips, & inquiries in JGIS¶
In this notebook, we assume that you went through all of the Basics examples at the very least. In the following notebook we offer a more in depth view of urban dynamics in Downtown Brooklyn by analysing (1) motor vehicle collisions, (2) taxi trip pickups, and (3) 311 NYC sidewalk inquiries.
Data source:
- https://data.cityofnewyork.us/Public-Safety/Motor-Vehicle-Collisions-Crashes/h9gi-nx95/
- https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page
- https://data.cityofnewyork.us/Social-Services/311-Service-Requests-from-2010-to-Present/erm2-nwe9/
More readings:
In [ ]:
Copied!
from urban_mapper import UrbanMapper
from urban_mapper.pipeline import UrbanPipeline
from urban_mapper.mixins.jupyter_gis import LayerStyle
from urban_mapper import UrbanMapper
from urban_mapper.pipeline import UrbanPipeline
from urban_mapper.mixins.jupyter_gis import LayerStyle
Initialise UrbanMapper¶
We initialise UrbanMapper to manage our pipelines and visualisation.
In [ ]:
Copied!
um = UrbanMapper()
um = UrbanMapper()
Reusing the Collision Pipeline¶
This section utilises the collision pipeline previously saved from the notebook located at:
examples/Study Cases/Downtown BK Collisions Study/[4]Downtown_BK_Collisions_Advanced_Pipeline_Extras.ipynb
Alternatively, you may reuse one of the following pipelines:
examples/Study Cases/Downtown BK Collisions Study/[3]Downtown_BK_Collisions_Advanced_Pipeline.ipynbexamples/Study Cases/Downtown BK Collisions Study/[2]Downtown_BK_Collisions_Pipeline.ipynb
Instructions¶
To reuse a pipeline:
- Open and execute the desired notebook.
- Invoke the
savemethod on the composed pipeline object. - This will generate a serialised file (e.g.,
name_of_pipeline.dill) containing the pipeline for later use, such as in this next cell.
Note¶
You also can create your pipline straight here in this jupyter notebook then pass it into the pipeline parameter of .with_pipeline(.) introduced afterward without any issue. .with_pipeline(.) deals with 1) pipeline saved, pipeline non-composed and pipeline already composed under the hood for you.
Here we show via an already-saved pipeline to gain time and show how fast it is.
Enjoy working with the pipeline!
Reusing the Taxi Trips Pipeline¶
This section utilises the taxi trips pipeline previously saved from the notebook located at:
examples/Study Cases/Downtown BK Taxi Trips Study/[4]Downtown_BK_Taxi_Trips_Advanced_Pipeline_Extras.ipynb
Alternatively, you may reuse one of the following pipelines:
examples/Study Cases/Downtown BK Taxi Trips Study/[2]Downtown_BK_Taxi_Trips_Pipeline.ipynbexamples/Study Cases/Downtown BK Taxi Trips Study/[3]Downtown_BK_Taxi_Trips_Advanced_Pipeline.ipynb
Instructions¶
To reuse a pipeline:
- Open and execute the desired notebook.
- Invoke the
savemethod on the composed pipeline object. - This will generate a serialised file (e.g.,
name_of_pipeline.dill) containing the pipeline for later use, such as in this next cell.
Note¶
You also can create your pipline straight here in this jupyter notebook then pass it into the pipeline parameter of .with_pipeline(.) introduced afterward without any issue. .with_pipeline(.) deals with 1) pipeline saved, pipeline non-composed and pipeline already composed under the hood for you.
Here we show via an already-saved pipeline to gain time and show how fast it is.
Enjoy working with the pipeline!
Define the Sidewalk Pipeline¶
First and foremost, the following is not a saved already pipeline per prior examples or study cases; hence we'll design it.
This pipeline processes 311 sidewalk inquiries, identifying the most common complaint type per sidewalk segment.
NYC311 in a nutshell: It responds to thousands of inquiries, comments and requests from customers every single day. This dataset represents only service requests that can be directed to specific agencies.
Be aware that this is a very very basic urban pipeline. We recommend to explore the Study cases folder examples for more advanced pipeline. Here only one enrichment is being performed, this is far from being advanced.
In [ ]:
Copied!
def most_common_inquiry(series):
if series.empty:
return None
mode = series.mode()
return mode.iloc[0] if not mode.empty else None
sidewalk_pipeline = UrbanPipeline(
[
(
"urban_layer",
(
um.urban_layer.with_type("streets_sidewalks")
.from_file("<path_to>/example-Network-04-03-2025_22_16.shp")
# The above's inference needs to be done via Tile2Net.
# Follow https://github.com/VIDA-NYU/tile2net for more details.
# https://github.com/VIDA-NYU/UrbanMapper/issues/17 discusses an enhancement to this.
.with_mapping(
longitude_column="Longitude",
latitude_column="Latitude",
output_column="nearest_sidewalk"
)
.build()
)
),
(
"loader",
(
um.loader.from_file("<path_to>/nyc_311.csv")
.with_columns(
longitude_column="Longitude",
latitude_column="Latitude"
)
.build()
)
),
(
"imputer",
(
um.imputer.with_type("SimpleGeoImputer")
.on_columns("Longitude", "Latitude")
.build()
)
),
("filter", um.filter.with_type("BoundingBoxFilter").build()),
(
"enricher",
(
um.enricher.with_data(group_by="nearest_sidewalk", values_from="Complaint Type")
.aggregate_by(method=most_common_inquiry, output_column="most_common_inquiry")
.build()
)
)
]
)
sidewalk_pipeline.preview()
def most_common_inquiry(series):
if series.empty:
return None
mode = series.mode()
return mode.iloc[0] if not mode.empty else None
sidewalk_pipeline = UrbanPipeline(
[
(
"urban_layer",
(
um.urban_layer.with_type("streets_sidewalks")
.from_file("/example-Network-04-03-2025_22_16.shp")
# The above's inference needs to be done via Tile2Net.
# Follow https://github.com/VIDA-NYU/tile2net for more details.
# https://github.com/VIDA-NYU/UrbanMapper/issues/17 discusses an enhancement to this.
.with_mapping(
longitude_column="Longitude",
latitude_column="Latitude",
output_column="nearest_sidewalk"
)
.build()
)
),
(
"loader",
(
um.loader.from_file("/nyc_311.csv")
.with_columns(
longitude_column="Longitude",
latitude_column="Latitude"
)
.build()
)
),
(
"imputer",
(
um.imputer.with_type("SimpleGeoImputer")
.on_columns("Longitude", "Latitude")
.build()
)
),
("filter", um.filter.with_type("BoundingBoxFilter").build()),
(
"enricher",
(
um.enricher.with_data(group_by="nearest_sidewalk", values_from="Complaint Type")
.aggregate_by(method=most_common_inquiry, output_column="most_common_inquiry")
.build()
)
)
]
)
sidewalk_pipeline.preview()
Define Visualisation Styles¶
Styles for all three layers:
- Collisions: Yellow to dark red (discrete).
- Taxi Pickups: Light yellow to dark red (linear).
- Sidewalk Inquiries: Categorical colors based on complaint type (e.g., purple for noise, hot pink for parking).
In [ ]:
Copied!
collision_style = LayerStyle(
attribute="collision_count",
stops={
50: [255, 255, 0, 1], # Yellow
200: [255, 215, 0, 1], # Gold
500: [255, 165, 0, 1], # Orange
1500: [255, 0, 0, 1], # Red
3000: [139, 0, 0, 1] # Dark red
},
interpolation_type="discrete",
default_value=[0, 0, 0, 1]
)
taxi_style = LayerStyle(
attribute="pickup_count",
stops={
100: [255, 255, 153, 1], # Bright light yellow
101: [255, 51, 153, 1], # Hot pink
2000: [153, 51, 255, 1], # Bright purple
5000: [139, 0, 0, 1] # Dark red
},
interpolation_type="linear"
)
sidewalk_style = LayerStyle(
attribute="most_common_inquiry",
stops={
"Noise": [128, 0, 128, 1], # Purple
"Noise - Commercial": [128, 0, 128, 1],
"Noise - Residential": [128, 0, 128, 1],
"Noise - Street/Sidewalk": [128, 0, 128, 1],
"Noise - Vehicle": [128, 0, 128, 1],
"Illegal Parking": [255, 105, 180, 1], # Hot pink
"Blocked Driveway": [255, 105, 180, 1],
"Abandoned Bike": [255, 105, 180, 1],
"Abandoned Vehicle": [255, 105, 180, 1],
"Derelict Vehicles": [255, 105, 180, 1],
"Broken Parking Meter": [255, 105, 180, 1],
"Bench": [169, 169, 169, 1], # Gray
"Curb Condition": [169, 169, 169, 1],
"Sidewalk Condition": [169, 169, 169, 1],
"Street Condition": [169, 169, 169, 1],
"Street Light Condition": [169, 169, 169, 1],
"Traffic Signal Condition": [169, 169, 169, 1],
"Root/Sewer/Sidewalk Condition": [169, 169, 169, 1],
"Obstruction": [169, 169, 169, 1],
"Dirty Condition": [173, 255, 47, 1], # Green yellow
"Sewer": [173, 255, 47, 1],
"Rodent": [173, 255, 47, 1],
"Dumpster Complaint": [173, 255, 47, 1],
"Illegal Dumping": [173, 255, 47, 1],
"Missed Collection": [173, 255, 47, 1],
"Commercial Disposal Complaint": [173, 255, 47, 1],
"APPLIANCE": [0, 0, 255, 1], # Blue
"Boilers": [0, 0, 255, 1],
"ELECTRIC": [0, 0, 255, 1],
"PLUMBING": [0, 0, 255, 1],
"GENERAL": [0, 0, 255, 1],
"HEAT/HOT WATER": [0, 0, 255, 1],
"UNSANITARY CONDITION": [0, 0, 255, 1],
"DOOR/WINDOW": [0, 0, 255, 1],
"Elevator": [0, 0, 255, 1],
"General Construction/Plumbing": [0, 0, 255, 1],
"Indoor Air Quality": [0, 0, 255, 1],
"Non-Residential Heat": [0, 0, 255, 1],
"Animal in a Park": [0, 128, 0, 1], # Green
"Dead Animal": [0, 128, 0, 1],
"Drug Activity": [139, 69, 19, 1], # Saddle brown
"Homeless Person Assistance": [139, 69, 19, 1],
"Encampment": [139, 69, 19, 1],
"Emergency Response Team (ERT)": [139, 69, 19, 1],
"Lead": [139, 69, 19, 1],
"Food Poisoning": [139, 69, 19, 1],
"Panhandling": [139, 69, 19, 1],
"Consumer Complaint": [255, 215, 0, 1], # Gold
"Food Establishment": [255, 215, 0, 1],
"Mobile Food Vendor": [255, 215, 0, 1],
"Vendor Enforcement": [255, 215, 0, 1],
"For Hire Vehicle Complaint": [255, 215, 0, 1],
"Taxi Complaint": [255, 215, 0, 1],
"Taxi Report": [255, 215, 0, 1],
"Graffiti": [0, 255, 255, 1], # Cyan
"Day Care": [64, 224, 208, 1], # Turquoise
"Maintenance or Facility": [64, 224, 208, 1],
"Borough Office": [64, 224, 208, 1],
"Non-Emergency Police Matter": [64, 224, 208, 1],
"Water System": [64, 224, 208, 1],
"Water Conservation": [64, 224, 208, 1],
"Snow or Ice": [64, 224, 208, 1],
"Other": [139, 0, 0, 1] # Dark red
},
interpolation_type="exact",
default_value=[169, 169, 169, 1]
)
collision_style = LayerStyle(
attribute="collision_count",
stops={
50: [255, 255, 0, 1], # Yellow
200: [255, 215, 0, 1], # Gold
500: [255, 165, 0, 1], # Orange
1500: [255, 0, 0, 1], # Red
3000: [139, 0, 0, 1] # Dark red
},
interpolation_type="discrete",
default_value=[0, 0, 0, 1]
)
taxi_style = LayerStyle(
attribute="pickup_count",
stops={
100: [255, 255, 153, 1], # Bright light yellow
101: [255, 51, 153, 1], # Hot pink
2000: [153, 51, 255, 1], # Bright purple
5000: [139, 0, 0, 1] # Dark red
},
interpolation_type="linear"
)
sidewalk_style = LayerStyle(
attribute="most_common_inquiry",
stops={
"Noise": [128, 0, 128, 1], # Purple
"Noise - Commercial": [128, 0, 128, 1],
"Noise - Residential": [128, 0, 128, 1],
"Noise - Street/Sidewalk": [128, 0, 128, 1],
"Noise - Vehicle": [128, 0, 128, 1],
"Illegal Parking": [255, 105, 180, 1], # Hot pink
"Blocked Driveway": [255, 105, 180, 1],
"Abandoned Bike": [255, 105, 180, 1],
"Abandoned Vehicle": [255, 105, 180, 1],
"Derelict Vehicles": [255, 105, 180, 1],
"Broken Parking Meter": [255, 105, 180, 1],
"Bench": [169, 169, 169, 1], # Gray
"Curb Condition": [169, 169, 169, 1],
"Sidewalk Condition": [169, 169, 169, 1],
"Street Condition": [169, 169, 169, 1],
"Street Light Condition": [169, 169, 169, 1],
"Traffic Signal Condition": [169, 169, 169, 1],
"Root/Sewer/Sidewalk Condition": [169, 169, 169, 1],
"Obstruction": [169, 169, 169, 1],
"Dirty Condition": [173, 255, 47, 1], # Green yellow
"Sewer": [173, 255, 47, 1],
"Rodent": [173, 255, 47, 1],
"Dumpster Complaint": [173, 255, 47, 1],
"Illegal Dumping": [173, 255, 47, 1],
"Missed Collection": [173, 255, 47, 1],
"Commercial Disposal Complaint": [173, 255, 47, 1],
"APPLIANCE": [0, 0, 255, 1], # Blue
"Boilers": [0, 0, 255, 1],
"ELECTRIC": [0, 0, 255, 1],
"PLUMBING": [0, 0, 255, 1],
"GENERAL": [0, 0, 255, 1],
"HEAT/HOT WATER": [0, 0, 255, 1],
"UNSANITARY CONDITION": [0, 0, 255, 1],
"DOOR/WINDOW": [0, 0, 255, 1],
"Elevator": [0, 0, 255, 1],
"General Construction/Plumbing": [0, 0, 255, 1],
"Indoor Air Quality": [0, 0, 255, 1],
"Non-Residential Heat": [0, 0, 255, 1],
"Animal in a Park": [0, 128, 0, 1], # Green
"Dead Animal": [0, 128, 0, 1],
"Drug Activity": [139, 69, 19, 1], # Saddle brown
"Homeless Person Assistance": [139, 69, 19, 1],
"Encampment": [139, 69, 19, 1],
"Emergency Response Team (ERT)": [139, 69, 19, 1],
"Lead": [139, 69, 19, 1],
"Food Poisoning": [139, 69, 19, 1],
"Panhandling": [139, 69, 19, 1],
"Consumer Complaint": [255, 215, 0, 1], # Gold
"Food Establishment": [255, 215, 0, 1],
"Mobile Food Vendor": [255, 215, 0, 1],
"Vendor Enforcement": [255, 215, 0, 1],
"For Hire Vehicle Complaint": [255, 215, 0, 1],
"Taxi Complaint": [255, 215, 0, 1],
"Taxi Report": [255, 215, 0, 1],
"Graffiti": [0, 255, 255, 1], # Cyan
"Day Care": [64, 224, 208, 1], # Turquoise
"Maintenance or Facility": [64, 224, 208, 1],
"Borough Office": [64, 224, 208, 1],
"Non-Emergency Police Matter": [64, 224, 208, 1],
"Water System": [64, 224, 208, 1],
"Water Conservation": [64, 224, 208, 1],
"Snow or Ice": [64, 224, 208, 1],
"Other": [139, 0, 0, 1] # Dark red
},
interpolation_type="exact",
default_value=[169, 169, 169, 1]
)
Create the Interactive Map¶
The map combines all three layers:
- Collisions as circles.
- Taxi pickups as lines.
- Sidewalk inquiries as lines (adjust
typeif your data uses points or polygons).
In [ ]:
Copied!
jgis_analysis, jgis_doc = (
um.jupyter_gis.with_document_settings(zoom=15)
.with_raster_layer(
url="http://basemaps.cartocdn.com/dark_all/{z}/{x}/{y}.png",
name="Dark Base Map",
attribution="© OpenStreetMap contributors",
opacity=0.9
)
.with_pipeline(
pipeline=sidewalk_pipeline,
layer_name="Sidewalk Inquiries",
layer_style=sidewalk_style,
opacity=0.8,
type="line"
)
.with_pipeline(
pipeline="<path_to>/taxi_advanced_pipeline.dill",
layer_name="Taxi Pickup Density",
layer_style=taxi_style,
opacity=0.6,
type="line"
)
.with_pipeline(
pipeline="<path_to>/collisions_advanced_pipeline.dill",
layer_name="Collision Hotspots",
layer_style=collision_style,
opacity=0.7,
type="circle"
)
.build()
)
jgis_analysis, jgis_doc = (
um.jupyter_gis.with_document_settings(zoom=15)
.with_raster_layer(
url="http://basemaps.cartocdn.com/dark_all/{z}/{x}/{y}.png",
name="Dark Base Map",
attribution="© OpenStreetMap contributors",
opacity=0.9
)
.with_pipeline(
pipeline=sidewalk_pipeline,
layer_name="Sidewalk Inquiries",
layer_style=sidewalk_style,
opacity=0.8,
type="line"
)
.with_pipeline(
pipeline="/taxi_advanced_pipeline.dill",
layer_name="Taxi Pickup Density",
layer_style=taxi_style,
opacity=0.6,
type="line"
)
.with_pipeline(
pipeline="/collisions_advanced_pipeline.dill",
layer_name="Collision Hotspots",
layer_style=collision_style,
opacity=0.7,
type="circle"
)
.build()
)
Save the Map¶
We save the comprehensive map as a .jGIS file.
DISCLAIMER: A file on the left will be created. Make sure to open it, and tada!
In the meantime, you can play with the jgis_doc directly in this notebook if of interest too!
In [ ]:
Copied!
jgis_analysis.save("comprehensive_analysis.jGIS")
jgis_analysis.save("comprehensive_analysis.jGIS")
In [ ]:
Copied!