API Notifications Overview

Street Manager is offering near real-time notification of events relating to a work which occur in the service.

The API Notifications solution is being offered to provide integrators a way to receive near real-time updates as events occur on a works record. Any eligible event will trigger a notification to be sent containing information on what event occurred. An eligible event will be any event which occurs relating to a work record e.g. Work start, Permit submitted, Inspection outcome accepted, Section 74 warning issued etc.

The API Notifications solution should be used as the default polling solution for API users, rather than the existing Event API GET works/updates endpoint. NOTE: The GET works/updates endpoint will continue to exist to allow for retrieval of historical data and for onboarding new users to your integration but should not be used for normal day to day activity. It is our intention to change the GET works/updates endpoint to limit/restrict its current functionality.

Approach

A publisher/subscriber model using Amazon Web Service’s (AWS) Simple Notification Service sends notifications to subscribers when an applicable event occurs in Street Manager.

Users subscribing to the service will be required to host a POST endpoint capable of receiving HTTPS requests from AWS’s source IP range.

Once this endpoint is configured a verification message will be sent to subscribers containing a confirmation link. This link must be called by the subscriber in order to confirm their subscription before notifications can be received.

Once verified, any event against a work associated with your organisation or an organisation you contract on behalf of will trigger a POST request to your subscriber endpoint with the notification specification represented below.

Messages

You can see the event data message format in the Event Notifier Message JSON Schema.

As Street Manager continues to add new functionality, we expect new object_types and event_details to be added each phase. Integrations should be built in such as way to support new object_types and event_details as they are released. However, the structure of the message will remain static. NOTE: Street Manager reserves the right to make unannounced breaking changes to the message structure in the event it is deemed necessary.

Subscription Confirmation message

Received when subscription is set up.

We strongly recommend that you set up your endpoint to automatically confirm subscriptions, by calling the SubscribeURL. For an example of how to set up your HTTPS endpoint to confirm a subscription, see Example HTTP subscriber

The SubscribeURL will stay active for 48 hours, if the subscriber does not confirm within that period they will need to unsubscribe and re-subscribe using the Street Manager Event API endpoints.

In order to ensure only authorised messages are confirmed, validate that the topic ARN (Amazon Resource Name) matches the following, before calling the SubscribeURL:

arn:aws:sns:eu-west-2:287813576808:prod-event-topic

{
  "Type": "SubscriptionConfirmation",
  "MessageId": "GUID",
  "Token": "TOKEN",
  "TopicArn": "UNIQUE REFERENCE FOR SNS TOPIC",
  "Message": "You have chosen to subscribe to the topic <TOPIC ARN>\nTo confirm the subscription, visit the SubscribeURL included in this message.",
  "SubscribeURL": "SUBSCRIPTION URL",
  "Timestamp": "2020-06-04T10:05:15.215Z",
  "SignatureVersion": "1",
  "Signature": "MESSAGE SIGNATURE",
  "SigningCertURL": "https://sns.eu-west-2.amazonaws.com/SimpleNotificationService-a86cb10b4e1f29c941702d737128f7b6.pem"
}

API Notification messages

Received when an event occurs.
The Message property contains the details of the event
The MessageAttributes property can be used to filter messages by org_ref. This is outlined in further detail below

{
  "Type" : "Notification",
  "MessageId" : "GUID",
  "TopicArn" : "TOPIC ARN",
  "Message" : "{/"event_reference\":19,\"event_type\":\"EVENT"\",\"object_type\":\"work\",
                "event_details\":\"planned_works_record_created\",\"event_time\":\"2023-11-
                07T09:36:58.011Z\",\"object_reference\":\"000014137888\",\"workstream_prefix\":\"001
                \",\"work_reference_number\":\"000014137888\",\"username\":\"planner@sm.com\",
                \"version\":1}",
  "Timestamp" : "2023-11-07T09:36:58.011Z",
  "SignatureVersion" : "1",
  "Signature" : "MESSAGE SIGNATURE",
  "SigningCertURL" : "https://sns.eu-west-2.amazonaws.com/SimpleNotificationService-a86cb10b4e1f29c941702d737128f7b6.pem",
  "UnsubscribeURL" : "https://sns.eu-west-2.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:<TOPIC-ARN>:0d59a03a-d105-4da1-ae5f-875b99b3472e",
  "MessageAttributes" : {
    "org_ref" : {"Type":"String.Array","Value":"[\"5990\",\"STPR\"]"}
  }
}

How do filter policies work?

The MessageAttributes section of each API notification contains a list of org_refs. The SWA codes contained here correspond to the Highway authority and Promoter who are associated with the Work and may receive this notification, if subscriptions are confirmed. Contractor org_refs will not appear in this list, their subscriptions are created using the SWA code of Promoters who they have authority to contract on behalf of. You will only receive notifications for events relating to works where your organisation’s SWA code is the Highway authority or Promoter for those works or your organisation is a Contractor with an active contract for the Promoter associated with the works.

This section outlines how to subscribe to the event topic.

Configure a POST endpoint which is accessible by AWS's Simple Notification Service (SNS). See Example HTTP subscriber
Call the Street Manager Event API endpoint POST api-notifications/subscribe, as an authenticated user in the organisation you want to subscribe to, providing your HTTPS endpoint.
Subscriber makes a GET request to the SubscribeURL in the confirmation message from SNS
Subscriber verify message contents using the signature before processing
Subscriber processes messages

NOTE: Initially, subscriptions to the event topic will be limited to one per organisation.

How to unsubscribe

This section outlines how to unsubscribe from the event topic.

Call the Street Manager Event API endpoint POST api-notifications/unsubscribe, as an authenticated user in the organisation you want to unsubscribe from.
Subscription is removed from Amazon Web Service's SNS and the Street Manager database.
Subscriber no longer receives messages.

Subscription Caveats

This section outlines caveats around the API notification process users should be aware of.

In the event that a user does not confirm their subscription within 48 hours the subscription will be removed from SNS by Amazon Web Services. Street Manager has no oversight of this process.
1. If the above scenario occurs the Event API POST api-notifications/unsubscribe endpoint should be used before a new subscription can be set up.
An unsubscribe URL is provided by AWS in the API Notification message. However, we would ask integrators to instead use the POST api-notifications/unsubscribe endpoint to unsubcribe to ensure a new subscription can be added in the future.
When a contractor user creates a subscription Street Manager will automatically subscribe the contractor to all organisations they are contracted to. In addition if contracts are added/removed from a Promoter organisation after the Contractor has confirmed their subscription, Street Manager will automatically update the subscription to reflect this.
If a contractor user has not confirmed their subscription within the 48 hours, when contracts are added/removed Street Manager will detect that the subscription no longer exists on SNS and delete the subscription from the database allowing for a new subscription to be set up.
Contractors will receive notifications about all works associated with Promoters they contract for. This will include notifications on workstreams they do not have access too. Contractor organisations will be unable to get additional detail about these works and should filter these notifications out, as they currently do with the polling endpoint.

Retry Policy

This section outlines the retry policy in the event that a notification fails to send.

In the event that a notification fails to send or your subscription fails to receive a notification due to a service outage or for any other reason, Amazon SNS will attempt to resend the notification up to 20 times over the period of approximately one hour. Retries will be initially attempted rapidly before gradually increasing the delay between each retry attempt. Any messages not successful after that will be discarded.

In this event or in the event of a service outage to Street Manager in which no notifications are being sent, the Street Manager Event API GET works/updates endpoint may be used to retrieve missed events to bring your integration back inline with Street Manager. NOTE: In the future usage of and data returned from the GET works/updates endpoint may be more restrictive than at the time of writing.

Testing

Subscriptions should be tested fully in the Sandbox environment before subscribing to API Notifications in Production. To test your service you may subscribe your test environment URL to the Sandbox feed using the Event API POST api-notifications/subscribe endpoint on Sandbox.

Support

If you need support using API Notifications, please raise a request using the service desk.