Open Data Overview

Street Manager is offering near real-time notification of events which occur in the service.

Approach

A publisher/subscriber model using Amazon Web Service’s (AWS) Simple Notification Service sending notifications to subscribers when an applicable event occurs in Street Manager.

Users subscribing to the service will be required to host a POST endpoint capable of receiving HTTPS requests from AWS’s source IP range.

Once this endpoint is configured a verification message will be sent to subscribers containing a confirmation link. This link must be called by the subscriber in order to confirm their subscription.

Once the subscriber is verified any event will trigger a POST request to the specified subscriber endpoint with the notification specification represented below.

Subscribers can filter messages on their consuming service, e.g. Filter by highway_authority_swa_code or area property.

Events

Events available

The open data offering operates using an event driven model, this section describes the events which trigger an open data notification.

Events currently available

Work start
Work stop
Work start reverted
Work stop reverted
Permit submitted
Permit granted
Permit refused
Permit revoked
Permit cancelled
Permit alteration granted
Activity created
Activity cancelled
Activity updated
Section 58 created
Section 58 in-force
Section 58 cancelled
Section 58 closed
Current traffic management type updated

Messages

You can see the event data message format in the Event Notifier Message JSON Schema.

NOTE: This message specification is subject to extension/modification with no prior notice. We recommend that subscribers do not use a serialisier for the JSON message and instead only extract properties they require for forward compatibility.

Subscription Confirmation message

Received when subscription is set up for each topic (event type) E.g. Permit.

We strongly recommend that you set up your endpoint to automatically confirm subscriptions, by calling the SubscribeURL. For an example of how to set up your HTTPS endpoint to confirm a subscription, see Example HTTP subscriber

The SubscribeURL will stay active for 3 days, if the subscriber does not call it within that period they will need to re-register their interest in Open Data with DFT

In order to ensure only authorised messages are confirmed validate that the topic ARN (Amazon Resource Name) matches one of the following, before calling the SubscribeURL:

arn:aws:sns:eu-west-2:287813576808:prod-activity-topic for activities
arn:aws:sns:eu-west-2:287813576808:prod-permit-topic for permits
arn:aws:sns:eu-west-2:287813576808:prod-section-58-topic for section 58s

If the subscriber only wishes to recieve notifications from one of the topics simply ignore the confirmation message from the undesired topic and no additional messages will be recieved for that event stream.

{
  "Type": "SubscriptionConfirmation",
  "MessageId": "GUID",
  "Token": "TOKEN",
  "TopicArn": "UNIQUE REFERENCE FOR SNS TOPIC",
  "Message": "You have chosen to subscribe to the topic <TOPIC ARN>\nTo confirm the subscription, visit the SubscribeURL included in this message.",
  "SubscribeURL": "SUBSCRIPTION URL",
  "Timestamp": "2020-06-04T10:05:15.215Z",
  "SignatureVersion": "1",
  "Signature": "MESSAGE SIGNATURE",
  "SigningCertURL": "https://sns.eu-west-2.amazonaws.com/SimpleNotificationService-a86cb10b4e1f29c941702d737128f7b6.pem"
}

Notification messages

Received when an event occurs.
The Message property can be of type Permit/Activity notification message. Examples of these can be found in the sections below.
The MessageAttributes property can be used to filter messages by the following fields: area, ha_org, activity_type, promoter_org, usrn. These fields will also exist on the Message property and descriptions of each can be found in the Event Notifier Message JSON Schema.

{
  "Type" : "Notification",
  "MessageId" : "GUID",
  "TopicArn" : "TOPIC ARN",
  "Message" : "{\"event_reference\":678,\"event_type\":\"WORK_START\",\"object_data\":{\"work_reference_number\":\"0000218889274\",\"permit_reference_number\":\"0000218889274-01\",\"promoter_swa_code\":\"STPR\",\"promoter_organisation\":\"Smoke Test Promoter\",\"highway_authority\":\"CITY OF WESTMINSTER\",\"works_location_coordinates\":\"POINT(527155.3328125 182227.946386719)\",\"street_name\":\"CHURCH STREET\",\"area_name\":\"CHURCH STREET\",\"work_category\":\"Minor\",\"traffic_management_type\":\"Road closure\",\"proposed_start_date\":\"2020-06-23T23:00:00.000Z\",\"proposed_start_time\":null,\"proposed_end_date\":\"2020-06-27T23:00:00.000Z\",\"proposed_end_time\":null,\"actual_start_date_time\":\"2020-06-24T10:11:00.000Z\",\"actual_end_date_time\":null,\"work_status\":\"Works in progress\",\"usrn\":\"8400794\",\"highway_authority_swa_code\":\"5990\",\"work_category_ref\":\"minor\",\"traffic_management_type_ref\":\"road_closure\",\"work_status_ref\":\"in_progress\",\"activity_type\":\"Section 58\",\"is_ttro_required\":\"No\",\"is_covid_19_response\":\"No\",\"works_location_type\":\"Cycleway, Footpath\",\"permit_conditions\":\"NCT01a, NCT01b, NCT11a\",\"road_category\":\"3\",\"is_traffic_sensitive\":\"Yes\",\"is_deemed\":\"No\",\"permit_status\":\"permit_modification_request\",\"town\":\"LONDON\"},\"event_time\":\"2020-06-24T14:21:44.091Z\",\"object_type\":\"PERMIT\",\"object_reference\":\"0000218889274-01\",\"version\":1}",
  "Timestamp" : "2020-06-24T14:22:01.847Z",
  "SignatureVersion" : "1",
  "Signature" : "MESSAGE SIGNATURE",
  "SigningCertURL" : "https://sns.eu-west-2.amazonaws.com/SimpleNotificationService-a86cb10b4e1f29c941702d737128f7b6.pem",
  "UnsubscribeURL" : "https://sns.eu-west-2.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:<TOPIC-ARN>:0d59a03a-d105-4da1-ae5f-875b99b3472e",
  "MessageAttributes" : {
    "area" : {"Type":"String","Value":"CHURCH STREET"},
    "ha_org" : {"Type":"String","Value":"5990"},
    "activity_type" : {"Type":"String","Value":"Section 58"},
    "promoter_org" : {"Type":"String","Value":"STPR"},
    "usrn" : {"Type":"Number","Value":"8400794"}
  }
}

Permit notification message

Events: Work start, Work stop, Work start reverted, Work stop reverted, Permit submitted, Permit granted, Permit refused, Permit revoked, Permit cancelled, Permit alteration granted

Example:

{
  "event_reference": 529770,
  "event_type": "work-start",
  "object_data": {
    "work_reference_number": "TSR1591199404915",
    "permit_reference_number": "TSR1591199404915-01",
    "promoter_swa_code": "STPR",
    "promoter_organisation": "Smoke Test Promoter",
    "highway_authority": "CITY OF WESTMINSTER",
    "works_location_coordinates": "LINESTRING(501251.53 222574.64,501305.92 222506.65)",
    "street_name": "HIGH STREET NORTH",
    "area_name": "LONDON",
    "work_category": "Standard",
    "traffic_management_type": "Road closure",
    "proposed_start_date": "2020-06-10T00:00:00.000Z",
    "proposed_start_time": "2020-06-10T13:50:00.000Z",
    "proposed_end_date": "2020-06-12T00:00:00.000Z",
    "proposed_end_time": null,
    "actual_start_date_time": "2020-06-11T10:11:00.000Z",
    "actual_end_date_time": null,
    "work_status": "Works in progress",
    "usrn": "8401426",
    "highway_authority_swa_code": "5990",
    "work_category_ref": "standard",
    "traffic_management_type_ref": "road_closure",
    "work_status_ref": "in_progress",
    "activity_type": "Remedial works",
    "is_ttro_required": "No",
    "is_covid_19_response" : "No",
    "works_location_type": "Cycleway, Footpath",
    "permit_conditions": "NCT01a, NCT01b, NCT11a",
    "road_category": "3",
    "is_traffic_sensitive": "Yes",
    "is_deemed": "No",
    "permit_status": "permit_modification_request",
    "town": "LONDON",
    "collaborative_working": "Yes",
    "collaboration_type": "Other",
    "collaboration_type_ref": "other",
    "close_footway": "Yes, a pedestrian walkway will be provided"
    "close_footway_ref": "yes_provide_pedestrian_walkway",
    "current_traffic_management_type": "Multi-way signals",
    "current_traffic_management_type_ref": "multi_way_signals",
    "current_traffic_management_update_date": "2020-06-11T10:11:00.000Z"
  },
  "event_time": "2020-06-04T08:00:00.000Z",
  "object_type": "PERMIT",
  "object_reference": "TSR1591199404915-01",
  "version": 1
}

Activity notification message

Events: Activity created, Activity cancelled, Activity updated

Example:

{
  "event_reference": 529771,
  "event_type": "ACTIVITY_CREATED",
  "object_data": {
    "activity_reference_number": "ARN-5990-85775436",
    "usrn": "8401426",
    "street_name": "Fake Street",
    "town": "London",
    "area_name": "MARYLEBONE HIGH STREET",
    "road_category": "4",
    "activity_coordinates": "LINESTRING(501251.53 222574.64,501305.92 222506.65)",
    "activity_name": "London Marathon",
    "activity_type": "event",
    "activity_type_details": "Activity type details",
    "start_date": "2020-06-10T00:00:00.000Z",
    "start_time": "2020-06-10T14:30:00.000Z",
    "end_date": "2020-06-11T00:00:00.000Z",
    "end_time": "2020-06-11T09:00:00.000Z",
    "activity_location_type": "Cycleway, Footpath",
    "activity_location_description": "Description of activity location",
    "traffic_management_type": "road_closure",
    "traffic_management_required": "Yes",
    "collaborative_working": "Yes"
    "cancelled": "Yes"
    "highway_authority": "CITY OF WESTMINSTER",
    "highway_authority_swa_code": "5990",
  },
  "event_time": "2020-06-04T08:00:00.000Z",
  "object_type": "ACTIVITY",
  "object_reference": "ARN-5990-85775436",
  "version": 1
}

Section 58 notification message

Events: Section 58 created, Section 58 in-force, Section 58 cancelled, Section 58 closed

Example:

{
  "event_reference": 529771,
  "event_type": "SECTION_58_CREATED",
  "object_data": {
    "section_58_reference_number": "S58-5990-46589070",
    "section_58_coordinates": "LINESTRING(501251.53 222574.64,501305.92 222506.65)",
    "section_58_status": "proposed",
    "start_date": "2020-06-10T14:30:00.000Z",
    "end_date": "2020-06-11T09:00:00.000Z",
    "section_58_duration": "six_months",
    "section_58_extent": "whole_road",
    "section_58_location_type": "Cycleway, Footpath",
    "status_change_date": "2020-06-10T14:30:00.000Z",
    "highway_authority_swa_code": "5990",
    "highway_authority": "CITY OF WESTMINSTER",
    "usrn": "8401426",
    "street_name": "Fake Street",
    "area_name": "MARYLEBONE HIGH STREET",
    "town": "London"
  },
  "event_time": "2020-06-04T08:00:00.000Z",
  "object_type": "SECTION_58",
  "object_reference": "S58-5990-46589070",
  "version": 1
}

Registration and onboarding approach

This section outlines the onboarding approach.

Configure a POST endpoint which is accessible by AWS's Simple Notification Service (SNS). See Example HTTP subscriber
Register to Open Data by providing an endpoint following the instructions on the Open Data registration page
Accept Terms of Use
A subscriber is added to the relevant topics
Subscriber makes a GET request to the SubscribeURL in the confirmation message from SNS
Subscriber verify message contents using the signature before processing
Subscriber processes messages

Archived open data notifications

Historical open data notifications have been archived and can be viewed here. Archived open data notifications.

Testing

All subscriptions use production environment events from the live service. To test your service you may subscribe your test environment URL to the feed.

Support

If you need support using the Open Data API, please raise a request using the service desk.

Stopping open data streaming

If you would like to stop recieving Open Data, please raise a request using the service desk.