Given a set of raster layers, it may be desirable to combine and filter the content of those layers. This is the function of map algebra. Two classes of map algebra operations are provided by GeoPySpark: local and focal operations. Local operations individually consider the pixels or cells of one or more rasters, applying a function to the corresponding cell values. For example, adding two rasters’ pixel values to form a new layer is a local operation.
Focal operations consider a region around each pixel of an input raster and apply an operation to each region. The result of that operation is stored in the corresponding pixel of the output raster. For example, one might weight a 5x5 region centered at a pixel according to a 2d Gaussian to effect a blurring of the input raster. One might consider this roughly equivalent to a 2d convolution operation.
Note: Map algebra operations work only on
and if a local operation requires multiple inputs, those inputs must
have the same layout and projection.
Before begining, all examples in this guide need the following boilerplate code:
import geopyspark as gps import numpy as np from pyspark import SparkContext from shapely.geometry import Point, MultiPolygon, LineString, box conf = gps.geopyspark_conf(master="local[*]", appName="map-algebra") pysc = SparkContext(conf=conf) # Setting up the data cells = np.array([[[3, 4, 1, 1, 1], [7, 4, 0, 1, 0], [3, 3, 7, 7, 1], [0, 7, 2, 0, 0], [6, 6, 6, 5, 5]]], dtype='int32') extent = gps.ProjectedExtent(extent = gps.Extent(0, 0, 5, 5), epsg=4326) layer = [(extent, gps.Tile.from_numpy_array(numpy_array=cells))] rdd = pysc.parallelize(layer) raster_layer = gps.RasterLayer.from_numpy_rdd(gps.LayerType.SPATIAL, rdd) tiled_layer = raster_layer.tile_to_layout(layout=gps.LocalLayout(tile_size=5))
Local operations on
TiledRasterLayers can use
floats, or other
abs are all of the local operations that currently supported.
(tiled_layer + 1) (2 - (tiled_layer * 3)) ((tiled_layer + tiled_layer) / (tiled_layer + 1)) abs(tiled_layer) 2 ** tiled_layer
Pyramid can also be used in local
operations. The types that can be used in local operations with
Note: Like with
TiledRasterLayer, performing calculations on
TiledRasterLayers means they must all
have the same layout and projection.
# Creating out Pyramid pyramid = tiled_layer.pyramid() pyramid + 1 (pyramid - tiled_layer) * 2
Focal operations are performed in GeoPySpark by executing a given
operation on a neighborhood throughout each tile in the layer. One can
select a neighborhood to use from the
Neighborhood enum class.
Likewise, an operation can be choosen from the enum class,
# This creates an instance of Square with an extent of 1. This means that # each operation will be performed on a 3x3 # neighborhood. ''' A square neighborhood with an extent of 1. o = source cell x = cells that fall within the neighbhorhood x x x x o x x x x ''' square = gps.Square(extent=1)
Miscellaneous Raster Operations¶
There are other means to extract information from rasters and to create rasters that need to be presented. These are polygonal summaries, cost distance, and rasterization.
Polygonal Summary Methods¶
In addition to local and focal operations, polygonal summaries can also
be performed on
TiledRasterLayers. These are operations that are
executed in the areas that intersect a given geometry and the layer.
Note: It is important the given geometry is in the same projection as the layer. If they are not, then either incorrect and/or only partial results will be returned.
poly_min = box(0.0, 0.0, 1.0, 1.0) tiled_layer.polygonal_min(geometry=poly_min, data_type=int)
poly_max = box(1.0, 0.0, 2.0, 2.5) tiled_layer.polygonal_min(geometry=poly_max, data_type=int)
poly_sum = box(0.0, 0.0, 1.0, 1.0) tiled_layer.polygonal_min(geometry=poly_sum, data_type=int)
poly_max = box(1.0, 0.0, 2.0, 2.0) tiled_layer.polygonal_min(geometry=poly_max, data_type=int)
cost_distance() is an iterative
method for approximating the weighted distance from a raster cell to a given
cost_distance function takes in a geometry and a
“friction layer” which essentially describes how difficult it is to traverse
each raster cell. Cells that fall within the geometry have a final cost of
zero, while friction cells that contain noData values will correspond to
noData values in the final result. All other cells have a value that describes
the minimum cost of traversing from that cell to the geometry. If the friction
layer is uniform, this function approximates the Euclidean distance, modulo some
cost_distance_cells = np.array([[[1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 1.0], [1.0, 1.0, 1.0, 1.0, 0.0]]]) tile = gps.Tile.from_numpy_array(numpy_array=cost_distance_cells, no_data_value=-1.0) cost_distance_extent = gps.ProjectedExtent(extent=gps.Extent(xmin=0.0, ymin=0.0, xmax=5.0, ymax=5.0), epsg=4326) cost_distance_layer = [(cost_distance_extent, tile)] cost_distance_rdd = pysc.parallelize(cost_distance_layer) cost_distance_raster_layer = gps.RasterLayer.from_numpy_rdd(gps.LayerType.SPATIAL, cost_distance_rdd) cost_distance_tiled_layer = cost_distance_raster_layer.tile_to_layout(layout=gps.LocalLayout(tile_size=5)) gps.cost_distance(friction_layer=cost_distance_tiled_layer, geometries=[Point(0.0, 5.0)], max_distance=144000.0)
It may be desirable to convert vector data into a raster layer. For
this, we provide the
function, which determines the set of pixel values covered by each vector
element, and assigns a supplied value to that set of pixels in a target raster.
If, for example, one had a set of polygons representing counties in the US, and
a value for, say, the median income within each county, a raster could be made
representing these data.
rasterize function can take a
(shapely.geometry), or a
PythonRDD[shapely.geometry]. These geometries will be
converted to rasters, then tiled to a given layout, and then be returned as a
TiledRasterLayer which contains these tiled values.
raster_poly_1 = box(0.0, 0.0, 5.0, 10.0) raster_poly_2 = box(3.0, 6.0, 15.0, 20.0) raster_poly_3 = box(13.5, 17.0, 30.0, 20.0) raster_multi_poly = MultiPolygon([raster_poly_1, raster_poly_2, raster_poly_3])
# Creates a TiledRasterLayer with a CRS of EPSG:4326 at zoom level 5. gps.rasterize(geoms=[raster_multi_poly], crs=4326, zoom=5, fill_value=1)
Rasterize a PythonRDD of Polygons¶
poly_rdd = pysc.parallelize([raster_poly_1, raster_poly_2, raster_poly_3]) # Creates a TiledRasterLayer with a CRS of EPSG:3857 at zoom level 5. gps.rasterize(geoms=poly_rdd, crs=3857, zoom=3, fill_value=10)
line_1 = LineString(((0.0, 0.0), (0.0, 5.0))) line_2 = LineString(((7.0, 5.0), (9.0, 12.0), (12.5, 15.0))) line_3 = LineString(((12.0, 13.0), (14.5, 20.0)))
# Creates a TiledRasterLayer whose cells have a data type of int16. gps.rasterize(geoms=[line_1, line_2, line_3], crs=4326, zoom=3, fill_value=2, cell_type=gps.CellType.INT16)
Rasterize Polygons and LineStrings¶
# Creates a TiledRasterLayer from both LineStrings and MultiPolygons gps.rasterize(geoms=[line_1, line_2, line_3, raster_multi_poly], crs=4326, zoom=5, fill_value=2)