Syncing an Entity to an External System
Syncing an Entity to an External System
A common use of the Lightspeed Retail (X-Series) API is to synchronize entities, like customers or products for example, to an external system. In this tutorial we will assume you are wanting to sync customers to the external system.
What you'll need
Please install Postman, or alternatively, use your favourite software development environment.
With simplicity in mind, we will use a GUI to connect to Lightspeed Retail (X-Series)'s API for this tutorial. Please feel free to use a software development platform instead, but this tutorial will use a GUI tool called Postman.
You should also know how to connect to Lightspeed Retail (X-Series)'s API. See the Quick Start tutorial.
You should also know how to connect to Lightspeed Retail (X-Series)’s API. See the Quick Start tutorial.
Overview
- Key concerns
- Understanding the Lightspeed Retail (X-Series) Data Structures
- Mapping entities
- Why versions
- Documentation
- Keeping it up to date
- What Next?
Key concerns
When synchronizing entities from Lightspeed Retail (X-Series) to an external system, it is important to understand the structure of the data that is returned from the Lightspeed Retail (X-Series) system so that you can successfully map it to the structure in the external system. You also want to ensure that you do not duplicate data in the external system so make sure you have a clear understanding of how you are going to do this. In the case of customers, for example, the Lightspeed Retail (X-Series) id is a UUID. If you can store that on the record in the external system, that will be ideal, but sometimes that is not possible, and you have to find an alternative way of mapping a Lightspeed Retail (X-Series) entity to the entity in the external system. We would highly recommend you document these mapping clearly so that both your team mates and your future self will know what you have done and why.
Understanding the Lightspeed Retail (X-Series) data structures
Most of the fields in the returned Lightspeed Retail (X-Series) data structures are self-explanatory, so generally only fields that have special values are documented. Given this, the easiest way to get to understand the Lightspeed Retail (X-Series) data structures is to use Postman (or your favourite develoment tool) and query for the data. So let us look at the customer data structure.
- Create a new Request within your collection.
GET,https://<<domain_prefix>>.retail.lightspeed.app/api/2.0/customers. Click Send.
This should return a data structure that looks similar to this:
"data": [
{
"id": "0242ac11-0002-11ea-e26d-7faf076fea1b",
"customer_code": "Customer-AVIP",
"first_name": "Customer A",
"last_name": "VIP",
"email": "myemail@<<domain_prefix>>.retail.lightspeed.app",
"year_to_date": 0.00000,
"balance": 0.000,
"loyalty_balance": 0.00000,
"note": null,
"gender": "M",
"date_of_birth": "1988-10-19",
"company_name": null,
"do_not_email": false,
"loyalty_email_sent": false,
"phone": "64-9-407-4472",
"mobile": null,
"fax": null,
"twitter": null,
"website": null,
"physical_suburb": "City",
"physical_city": "Auckland",
"physical_postcode": "0100",
"physical_state": "Auckland",
"postal_suburb": null,
"postal_city": null,
"postal_state": null,
"customer_group_id": "28020918-a068-aae4-11e9-cad0d053ea28",
"enable_loyalty": false,
"created_at": "2020-04-16T06:53:50+00:00",
"updated_at": "2020-04-28T01:47:13+00:00",
"deleted_at": null,
"customer_group_ids": [],
"version": 158291,
"postal_postcode": null,
"name": "Customer A VIP",
"physical_address_1": "33 retailer st",
"physical_address_2": null,
"physical_country_id": "NZ",
"postal_address_1": null,
"postal_address_2": null,
"postal_country_id": null,
"custom_field_1": "Custom data one",
"custom_field_2": null,
"custom_field_3": null,
"custom_field_4": null,
"time_until_deletion": null
},
...
],
"version": {
"min": 158291,
"max": 158306
}
]
As mentioned above, the first thing to note is that the id for a customer is a UUID. The second thing to note is the minimum and maximum version numbers at the bottom of the list. These are important and we will come back to them after looking at how we are going to do the mapping between the Lightspeed Retail (X-Series) data structure and our external system.
Mapping entities
Once you have an example output from the Lightspeed Retail (X-Series) system, you can start to look at how to map that to the data structures you have in the external system. If you are mapping customers then you would hope there will be a significant overlap between the two structures, but there may be complications, for example, if the customer address is stored in a different structure to the other details about the customer in the external system. In this case you need you will need to do some more work to ensure the mapping is successful.
The other issue you may encounter is fields in the external system that are not present in the Lightspeed Retail (X-Series) customer data but are required in the external system. In this case you need to think about how you will populate this field, either by using a default value or by generating a value based off of other properties.
Sometimes the Lightspeed Retail (X-Series) data structure may not give you sufficient information. Take for example the customer group. In the customer payload this is returned with the id of the customer group. In the external system you may need the name, so in which case you are going to need to make another call to the Lightspeed Retail (X-Series) system to get the details of the customer group.
- Create a new Request within your collection.
GEThttps://<<domain_prefix>>.retail.lightspeed.app/api/2.0/customer_groups/:id. Click Send.
This should return a data structure that looks similar to this:
{
"includes": null,
"data": {
"id": "28020918-a068-aae4-11e9-cad0d05570c9",
"name": "VIP Customers",
"group_id": "1111111112",
"version": 150535,
"created_at": "2020-04-29T06:54:05+00:00",
"updated_at": "2020-04-29T06:54:05+00:00",
"deleted_at": null
}
}
You now have the name of the customer group. However, only the id is unique. The name may change. This is where you need to make a decision about whether this is going to be critical for your organisation and if it is, is there a way you can mitigate issues if the name is changed.
Once you have decided on a mapping you can now write the code and do the testing required to do the actually syncing.
Why versions?
The returned list will have a maximum of 200 entries. If you look at the parameters that can be passed to the customers end point you will notice that page_size is one of them. You may be very tempted to just increase the page size to get all the records but we do not recommend this for this two reasons. One is that you may hit issues with timeouts and secondly, some queries are limited in the Lightspeed Retail (X-Series) system to ensure one query does not result in the system being overwhelmed. This is where the version numbers come in.
Most APIs provide a date filtering option. The issue with this is that in systems with a large number of transactions happening, and the fact that those transactions may be coming from external integrations, it is possible to miss entries by using date filtering. To get around this Lightspeed Retail (X-Series) has moved to a versioning system. Any transaction that happens results in the versioning number being incremented by 1. This way, if you select all entries with version greater than 158306 in this case, you are guaranteed to get all the entries, no matter the date order in which they arrived in the system. This means that you can paginate without fear of losing any entries. It also guarantees, in the case of updates, that the updates will be applied in the same order that they arrived in the system. This ensures consistency with the Lightspeed Retail (X-Series) system as you won't have the issue of updates being applied out of order.
So, in practise, how do you do this? Now that you have made your first call to customers, the next time you want to get the next two hundred rows. To do this all you need to is to make a call to https://<<domain_prefix>>.retail.lightspeed.app/api/2.0/customers?after=158306
"data": [
{
"id": "0242ac11-0002-11ea-ee93-89e63769ed55",
"customer_code": "WALKIN",
"first_name": null,
"last_name": null,
"email": null,
"year_to_date": 0.00000,
"balance": 0.000,
"loyalty_balance": 0.00000,
"note": null,
"gender": null,
"date_of_birth": "1988-10-19",
"company_name": null,
"do_not_email": false,
"loyalty_email_sent": false,
"phone": null,
"mobile": null,
"fax": null,
"twitter": null,
"website": null,
"physical_suburb": null,
"physical_city": null,
"physical_postcode": null,
"physical_state": null,
"postal_suburb": null,
"postal_city": null,
"postal_state": null,
"customer_group_id": "28020918-a068-aae4-11e9-cad0d05570c9",
"enable_loyalty": false,
"created_at": "2020-04-29T06:54:05+00:00",
"updated_at": "2020-04-30T01:16:28+00:00",
"deleted_at": null,
"customer_group_ids": [],
"version": 158307,
"postal_postcode": null,
"name": null,
"physical_address_1": null,
"physical_address_2": null,
"physical_country_id": null,
"postal_address_1": null,
"postal_address_2": null,
"postal_country_id": null,
"custom_field_1": null,
"custom_field_2": null,
"custom_field_3": null,
"custom_field_4": null,
"time_until_deletion": null
}
],
"version": {
"min": 158307,
"max": 158307
}
}
Notice that in this case the minimum and maximum version numbers are the same. This means that this is the last record. If you try to retrieve the records after 158307 then you will get an empty record set with the version numbers set to null, like this.
{
"data": [],
"version": {
"min": null,
"max": null
}
}
Paginating through the Results
As mentioned above, he results are returned with a default pagination size of 200, i.e. not more than 200 results will be returned at a time. Once you have the results it is a simple case of iterating over the data until you reach the end. You can then retrieve the next 200 results. In pseudocode you would do something like this:
retrieve(last_version_number)
version_number = last_version_number
while version_number is not null {
customers = call(`https://<<domain_prefix>>.retail.lightspeed.app/api/2.0/customers?after={version_number}`)
for customer in customers['data'] {
success = call process(customer)
if !success {
return error
}
}
version_number = customers['version']['max']
if version_number is not null {
last_version_number = version_number
save(last_version_number)
}
}
Note: Set the last_version_number to 0 to iterate over all records.
Keeping it up to date
he main question to be answered here is how quickly do you want the data to be updated. If almost immediate updates are required then we would recommend the use of webhooks, for which a tutorial is available here. However, if hourly or more is sufficient then a scheduled job that queries the Lightspeed Retail (X-Series) system for all entries with a version greater than the maximum of the last update is sufficent and easier. If you have stored the last version as per the pseudocode above it is easy to get all the new customers that have been added.
Note: If you are willing to pay for the syncing then, if your external system is supported, you may want to look at Zapier. Please contact us if this option is something you would like to investigate. Please note that Lightspeed Retail (X-Series) is not in anyway associated with Zapier, so we cannot provide support on Zapier integrations.
Documentation
Unless the mapping is essentially 1-1 and easy, we strongly recommend that you document how you have done the mapping and why you have made the decisions you have made. For starters, you probably won't remember next time you have to modify the mapping, and secondly, it will make the job of the next person who has to maintain it much easier if they know why certain decisions have been taken, as almost certainly, you will be on holiday at the time they need to change it.
What next?
Now that you have successfully synced customers, you can start to look at some of the more complex entities, like products and sales. The same principles apply.
Updated 8 months ago