Algo Store API
Contents
Introduction
An node express app for serving recommended rankings based on various algorithms.
This guide describes how to upload product catalogues and capture events needed to generate recommended rankings
Deployment
How is this deployed?
Use the teamcity steps
Where is this deployed?
This can be view as a service named default
in the services page. For the version history of the deployments:
Tests
To run the tests for this project, simply npm install
in this folder and in ./algorithms/simpleEventAnalysis
and then follow the step for your os.
Run-AllTests.ps1
npm test
Example site
You can run up some sites as an example
npm run example
This will start up:
- Algo Store Api
- Example site (like broadbandchoices)
- Provider site (like bt.com)
- Cloud function fake (like a google cloud function)
You should be able to test the whole thing out there.
CORS Exceptions
We need to configure CORS exceptions in the algo-store-api so that platform can retrieve a recommended sort order as well as collect events to help train the algorithm.
The standard url pattern that is configured for CORS exceptions is of the form partner.homecomms.env.dtlservices.co.uk
with an optional additional domain. You can see this is configured by adding an entry to the array passed into the generate
method in algoStoreApi which is defined in corsUrls. For any partners following non-standard urls we simply hardcode
those urls into the collection defined in corsUrls.
Once you have made your changes you simply need to deploy this api using the algo store api teamcity step.
After deploying your changes you can check if they're working because you will stop seeing errors such as the following (this example was generated from the clooper site) occurring in the console log in developer tools in the browser:
Access to XMLHttpRequest at 'https://dt-recommends.cdndtl.co.uk/events' from origin 'https://broadband.clooper.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource
Typically these errors occur when you load up the broadband homepage but it's also worthing checking the results page as well.
API Endpoints
- Uploading Products
- Retrieving Current Product Data
- Capturing events
- Getting recommended product orders
- Retrieving events
- Updating product orders
- Fullsight sales data
Uploading products
All of the products associated with events you will be adding can be uploaded in bulk. Every time the products on your website change the algo store will need to know about the changes to the products.
The full product catalogue can be uploaded by PUT
ing your products as follows
PUT
https://dt-recommends.cdndtl.co.uk/products
BODY
{
"products": [
{ "sku": 123, "price": 79.99, "colour": "red" },
{ "sku": 124, "price": 89.9, "colour": "blue" },
{ "sku": 125, "price": 49.9, "colour": "grey" }
],
"etag": "version_number"
}
The body must contain a products
array and an etag
string representing the product catalogue version.
products property
required Each product is an object containing a sku
and any other arbitrary data associated with the product.
etag property
required String representing product catalogue version
Retrieving the current product data
HEAD
https://dt-recommends.cdndtl.co.uk/products
Response will contain etag header containing last upload's version string
GET
https://dt-recommends.cdndtl.co.uk/products
Response body contains products data pushed during last upload
Capturing events
Include the client script on the site from which you would like to capture events somewhere in your html
<!-- AlgoStore script -->
<script>
window.algoStoreEvents = window.algoStoreEvents || []
window.algoStorePartnerId = "@Model.Tsid"; //Model is provided to view this script is included in
var script = document.createElement("script")
script.src = "https://dt-recommends.cdndtl.co.uk/js/algoStoreEvents.js"
script.async = true
document.head.appendChild(script)
</script>
<!-- AlgoStore script -->
This will add an algoStoreEvents
object to your window, used for adding events, and will also drop a client side accessible cookie on the browser named algoStoreSessionId.
The algoStoreEvents object has a single method, push
which can be called to log events. It accepts a single event.
Events must have a property called type
containing the type of the event, along with any other data you would like to capture
Events sent from the browser will automatically have the algoStoreSessionId
property added to the saved event, along with a created
time for the event (UNIX timestamp UTC). The following call from the client:
window.algoStoreEvents.push({
type: 'addToCart',
sku: 123,
onSent: function () {
console.log('fires after event is added')
})
})
Will result in a event that can be looked up later and used in your ranking algorithm:
{
type: 'addToCart',
sku: 123,
algoStoreSessionId: 'a6b4a0c1-4ef9-4175-b00c-13126fc0c607',
created: '1505721424',
partnerId: 'your-partner-id'
}
onSent onSent is an optional parameter that can be passed when adding an event. Pass it a function to be run after the event has been added.
Getting recommended product orders
To fetch the recommended product order as calculated by one of your algorithms, simply push an fetchRecommendedOrder
event to the window.algoStoreEvents
,
the algoId you want to sort by along with a callback function specifying what you want to do with the ordered product skus once they arrive:
window.algoStoreEvents.push({
type: 'fetchRecommendedOrder',
algoId: 'your-algo-id',
onRecommendedOrderFetched: function(error, orderedSkus) {
// sort your products based on the orderedSkus array
}
})
Retrieving events
GET
https://dt-recommends.cdndtl.co.uk/events
Response body contains all events pushed along with features from the product that the event was associated with
Optional querystring parameters startDate
and endDate
to filter the events returned
Updating product orders
Update recommended orders
POST
https://dt-recommends.cdndtl.co.uk/recommended-product-order
Body:
{
"algoId": "algorithmId",
"recommendedOrder": [12, 32, 8, 10]
}
Fullsight sales data
This is the endpoint which is used to update the Events table with sales data from Fullsight. The Data Team are usually responsible for ensuring that the Fullsight webhook is setup to send this data to this endpoint.
POST
https://dt-recommends.cdndtl.co.uk/fullsight-sales-event
Body:
{
"transaction_id": "350643684",
"sale_time": "2018-01-22T23:27:00.000Z",
"affiliate_network": "Affiliate Window",
"clickout_id": "d5b5e1c6-3842-432a-8611-ee8049d5302a",
"uid": "AQU4vegXwvyjD8",
"cid": "283971595.1516139071",
"partner_id": "94059",
"site_name": "broadbandchoices.partners",
"provider": "BT Broadband",
"product_name":
"BDB1 / BBO6 / FBR5 - Infinity Broadband New; BDB1 / BBO6 - Broadband New; TVO2 - TV New; BT Sport Provide; CAO17 - Calls Starter; TRP4 - TRIPLE PLAY PACKAGE",
"product_sku": "1202",
"product_category": "HomeComms",
"sale_status": "Confirmed",
"referral_ref": "P94059QU4vegXwvyjD8DTTR",
"revenue": "137.0"
}
The body accepts single events or arrays.
Troubleshooting
Problems Fetching Recommended Sort Order
Currently we don't have any datadog logging for the algo-store api so you need to check the GCP logs.