Algo Store API

An node express app for serving recommended rankings based on various algorithms.

This guide describes how to upload product catalogues and capture events needed to generate recommended rankings

Where is this deployed?

In google cloud, as a service named default. You can find this as follows (for live):

Go to the google cloud console -> right-click burger menu (top lhs) -> App Engine -> Service

Deploying

To sandbox

In powershell:

& ./Delploy-To-Sandbox.ps1

In bash:

./deploy-sandbox.sh

To live

In powershell:

& ./Delploy-To-Live.ps1

In bash:

./deploy.sh

Running the tests

To run the tests for this project, simply npm install in this folder and in ./algorithms/simpleEventAnalysis and then follow the step for your os.

Run-AllTests.ps1

npm test

Example site

You can run up some sites as an example

npm run example

This will start up:

• Algo Store Api
• Example site (like broadbandchoices)
• Provider site (like bt.com)
• Cloud function fake (like a google cloud function)

You should be able to test the whole thing out there.

Configuring CORS Exceptions

We need to configure CORS exceptions in the algo-store-api so that platform can retrieve a recommended sort order as well as collect events to help train the algorithm.

The standard url pattern that is configured for CORS exceptions is of the form partner.homecomms.env.dtlservices.co.uk with an optional additional domain. You can see this is configured by adding an entry to the array passed into the generate method in algoStoreApi which is defined in corsUrls. For any partners following non-standard urls we simply hardcode those urls into the collection defined in corsUrls.

Once you have made your changes you simply need to deploy this api using the algo store api teamcity step.

After deploying your changes you can check if they're working because you will stop seeing errors such as the following (this example was generated from the clooper site) occurring in the console log in developer tools in the browser:

Access to XMLHttpRequest at 'https://dt-recommends.cdndtl.co.uk/events' from origin 'https://broadband.clooper.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource

Typically these errors occur when you load up the broadband homepage but it's also worthing checking the results page as well.

Uploading products

All of the products associated with events you will be adding can be uploaded in bulk. Every time the products on your website change the algo store will need to know about the changes to the products.

The full product catalogue can be uploaded by PUTing your products as follows

PUT

https://dt-recommends.cdndtl.co.uk/products

BODY

{
  "products": [
    { "sku": 123, "price": 79.99, "colour": "red" },
    { "sku": 124, "price": 89.9, "colour": "blue" },
    { "sku": 125, "price": 49.9, "colour": "grey" }
  ],
  "etag": "version_number"
}

The body must contain a products array and an etag string representing the product catalogue version.

products property

required Each product is an object containing a sku and any other arbitrary data associated with the product.

etag property

required String representing product catalogue version

Retrieving the current product data

HEAD

https://dt-recommends.cdndtl.co.uk/products

Response will contain etag header containing last upload's version string

GET

https://dt-recommends.cdndtl.co.uk/products

Response body contains products data pushed during last upload

Capturing events

Include the client script on the site from which you would like to capture events somewhere in your html

<!-- AlgoStore script -->
<script>
window.algoStoreEvents = window.algoStoreEvents || []
window.algoStorePartnerId = "@Model.Tsid"; //Model is provided to view this script is included in
var script = document.createElement("script")
script.src = "https://dt-recommends.cdndtl.co.uk/js/algoStoreEvents.js"
script.async = true
document.head.appendChild(script)
</script>
<!-- AlgoStore script -->

This will add an algoStoreEvents object to your window, used for adding events, and will also drop a client side accessible cookie on the browser named algoStoreSessionId.

The algoStoreEvents object has a single method, push which can be called to log events. It accepts a single event.

Events must have a property called type containing the type of the event, along with any other data you would like to capture

Events sent from the browser will automatically have the algoStoreSessionId property added to the saved event, along with a created time for the event (UNIX timestamp UTC). The following call from the client:

window.algoStoreEvents.push({
  type: 'addToCart',
  sku: 123,
  onSent: function () {
    console.log('fires after event is added')
  })
})

Will result in a event that can be looked up later and used in your ranking algorithm:

{
  type: 'addToCart',
  sku: 123,
  algoStoreSessionId: 'a6b4a0c1-4ef9-4175-b00c-13126fc0c607',
  created: '1505721424',
  partnerId: 'your-partner-id'
}

onSent onSent is an optional parameter that can be passed when adding an event. Pass it a function to be run after the event has been added.

Getting recommended product orders

To fetch the recommended product order as calculated by one of your algorithms, simply push an fetchRecommendedOrder event to the window.algoStoreEvents, the algoId you want to sort by along with a callback function specifying what you want to do with the ordered product skus once they arrive:

window.algoStoreEvents.push({
  type: 'fetchRecommendedOrder',
  algoId: 'your-algo-id',
  onRecommendedOrderFetched: function(error, orderedSkus) {
    // sort your products based on the orderedSkus array
  }
})

Retrieving events

GET

https://dt-recommends.cdndtl.co.uk/events

Response body contains all events pushed along with features from the product that the event was associated with

Optional querystring parameters startDate and endDate to filter the events returned

Updating product orders

Update recommended orders

POST

https://dt-recommends.cdndtl.co.uk/recommended-product-order

Body:

{
  "algoId": "algorithmId",
  "recommendedOrder": [12, 32, 8, 10]
}

Fullsight sales data

Sales events from Fullsight by posting to the API as follows:

POST

https://dt-recommends.cdndtl.co.uk/fullsight-sales-event

Body:

{
  "transaction_id": "350643684",
  "sale_time": "2018-01-22T23:27:00.000Z",
  "affiliate_network": "Affiliate Window",
  "clickout_id": "d5b5e1c6-3842-432a-8611-ee8049d5302a",
  "uid": "AQU4vegXwvyjD8",
  "cid": "283971595.1516139071",
  "partner_id": "94059",
  "site_name": "broadbandchoices.partners",
  "provider": "BT Broadband",
  "product_name":
    "BDB1 / BBO6 / FBR5 - Infinity Broadband New; BDB1 / BBO6 - Broadband New; TVO2 - TV New; BT Sport Provide; CAO17 - Calls Starter; TRP4 - TRIPLE PLAY PACKAGE",
  "product_sku": "1202",
  "product_category": "HomeComms",
  "sale_status": "Confirmed",
  "referral_ref": "P94059QU4vegXwvyjD8DTTR",
  "revenue": "137.0"
}

The body accepts single events or arrays.