The project uses as reference data Geoscape Datasets for Australian roads and administrative boundaries. Over the years, I have worked with these datasets under commercial licence, but happily they are now available for public use under The Australian Government's Data and Digital Government Strategy. Let's take a look at them.

Geoscape National Roads

Note: National Roads © Geoscape Australia licensed by the Commonwealth of Australia under Creative Commons Attribution 4.0 International license (CC BY 4.0).

The Geoscape National Roads dataset is a digital representation of the road network of Australia, inclusive of bus ways, walking trails, cycleways, and even ferry routes. The dataset has rich metadata, inclusive of road hierarchy (e.g. national highway, arterial, local road) and travel direction. One facet I like is that each row – and its associated geometry – is defined intersection-to-intserection (e.g. an actual road might be broken into multiple rows, with each row being the segment between two intersections).

The open edition of this dataset varies subtly from the commercially licensed edition. Firstly, the metadata is almost (but not quite) identical:

• road_id – primary key
• contributor_id
• jurisdiction_control – for example Aboriginal Land Council, Botanical Gardens, Commonwealth, Council, Department of Environment and Conservation, Department of Transport, Dept of Defence, etc
• operator – for example Port Authority, SAG: Forestry SA, SAG: Water, Southern Rural Water, etc
• date_created, date_modified
• national_route, state_route
• full_street_name (upper case)
• street_name (upper case) / street_name_label (title case)
• street_type (upper case) / street_type_label (title case)
• street_suffix (upper case) / street_suffix_label (title case)
• street_alias_name (upper case) / street_alias_type (upper case) / street_alias_suffix (upper case)
• feature_type – one of DUAL CARRIAGEWAY, FERRY ROUTE, MOTORWAY, PATHWAY, RAMP, ROUNDABOUT, SINGLE CARRIAGEWAY, VEHICULAR_TRACK, or null
• hierarchy – one of ACCESS ROAD, ARTERIAL ROAD, BUSWAY, COLLECTOR ROAD, CYCLEPATH, FERRY, FOOTPATH, LOCAL ROAD, NATIONAL OR STATE HIGHWAY, SUB-ARTERIAL ROAD, VEHICLE TRACK, or null
• subtype – BRIDGE, FERRY, FIRE TRAIL, PATHWAY, RAMP, ROAD, ROUNDABOUT, TUNNEL, or null
• ground_relationship – ABOVE GROUND, BELOW GROUND, ON GROUND, or null
• lane_count – range 1-6 or null
• lane_description – ONE, TWO OR MORE, or null (note all three values apply where lane_count is null)
• one_way – ONE WAY, TWO WAY, null
• status – CLOSED, OPERATIONAL, PROPOSED, UNDER CONSTRUCTION, or null
• surface – SEALED, UNSEALED, or null
• trafficability – 2WD, 4WD, or null
• travel_direction – BOTH, FROM TO, TO FROM, null
• speed – 10 – 110, or null
• state, source – ACT, NSW, NT, QLD, SA, TAS, VIC, WA (note: no OT)
• horizontal_accuracy – range 1-250
• shape_length, shape

Secondly, there are more limited data formats and datums. My intention is to work with GDA2020 in GDB format.

Geoscape Administrative Boundaries

Note: Administrative Boundaries © Geoscape Australia licensed by the Commonwealth of Australia under Creative Commons Attribution 4.0 International license (CC BY 4.0).

The Geoscape Administrative Boundaries is a comprehensive set of national boundaries, including government, statistical and electoral boundaries.

The dataset comprises seven products:

• Localities
• Local Government Areas (LGAs)
• Wards
• Australian Bureau of Statistics (ABS) Boundaries
• Electoral Boundaries
• State Boundaries
• Town Points

The ABS Boundaries product is organised in three themes related to 2011, 2016 and 2021 boundary definitions. Of most interest to me is the 2021 ABS Mesh Blocks and Statistical Areas layer within the ABS Boundaries 2021 theme. This is described in detail on the ABS website product comprises:

• Australia
• States and Territories (S/T)
• Statistical Areas Level 4 (SA4s)
• Statistical Areas Level 3 (SA3s)
• Statistical Areas Level 2 (SA2s)
• Statistical Areas Level 1 (SA1s)
• Mesh Blocks (MBs)
• Greater Capital City Statistical Areas (GCCSAs) – not part of main structure

Like National Roads, this dataset is available only in limited datums and formats. I am going to work with GDA2020 downloaded in SHP format.

Tags: #TagHub #Geospatial #DuckDB

I do hope to curate this list at some stage, but at least I've now captured some of the links so I don't lose track of them.

Multi-Processing (e.g., concurrency, multi-threading, asynchrony) * Promises * Zap * Brad Cypert Blog * Programming Languages Memory Model * Making Sense of Acquire Release Semantics * Miguel Young Blog * Algorithms for Modern Hardware * Work-Stealing Deque Part 1: The Problem with Locks

Performance (e.g., algorithms, SIMD, branchless coding) * Daniel Lemire, Computer Science Professor * Ash's Blog * Tutorial on SIMD vectorisation to speed up brute force * Josh Haberman Blog * Latency Numbers Every Programmer Should Know * Optimizing UTC –> Unix Time Conversion for Size and Speed

Zig * Open My Mind – Karl Seguin Blog * Interfaces in Zig: Five Patterns You Should Know

Geospatial * Gamdev Maths: distance from point to line * Find the Intersection of Two Line Segments in 2D (Easy Method) * Fast geodesic approximations with Cheap Ruler (code is here) * Bing Maps Tile System note: interest here in quadkey spatial index * S2Geometry * An Algorithm for Polygon Intersections

Data Engineering * Jenna Jordan Blog * Data Engineering Blog of Simon Späti * Spartan Blog – Jerónimo * The Evolution of Database Architecture and the Future of Data Management * Stop Paying the Complexity Tax * Big Data is Dead