Sync
State: brainstorming and testing. I will implement it step by step before it becomes mature, so don't take this document as API docs...
As the frontend maintains its own local state to enable offline usage for core features, we need a sync mechanism. This is obviously tied to offline support concept. The sync system should be enough smart and provide a good experience, so it can run in the background every N minutes/hours without noticing it. When they are conflicts, the user should be informed rapidly to fix it. Conflicts come from double edition of the same element (i.e. 2 teachers modified the same exo), when the data validation is different on the server. In these cases, the interface should help the user to resolve conflicts and the error messages should be clear for each element to handle.
Sync interface
To facilitate the user experience depending on the use case, we have 2 parts for this interface.
The first part is the Sync panel in a dropdown opened via a cloud icon in the header strip. It includes:
- counters of elements to sync up in each category
- last sync date
- a button to manually sync
- a state indicator on the icon (sync up, sync down, offline, online)
- the state of loading with a sentence like "syncing down..." or "syncing up..."
TODO: do a mockup of this interface
The second interface is the Sync details MUST show the following data:
- counters and content of elements to sync up in each category
- content of new elements received during the sync down
- errors messages given from the API for each element to sync up
- last sync up and sync down status (success or fail)
TODO: do a mockup of this interface
General strategy
Data loading
Once the user logged in, the frontend fetch all courses data. When the user clicks on a course it loads all skills and exos and old answers related to this course. If there are elements on the selected course, new or updated elements will only be loaded via the Sync system.
Sync interval
The interval time is defined arbitrarily to 20 minutes.
TODO: really a good interval ?
Maybe:
- every 10 minutes answers are synced up if they exists
- every 30 minutes, it runs a sync down if the browser tab is active
Sync order
The frontend will regularly try to sync to save local state and get new data from the server. It first syncs down, to get a chance to resolve potential conflicts, then it syncs up.
TODO: how to manage incoherence or edge cases where the local state doesn't have all elements, but the last sync date is after their creation so there is no way to load them ?? Should we provide a button to force an entire reload ?
Sync down
On sync down the server will provide the following elements:
-
courses: new and updated courses -
skills: new and updated skills -
exos: new and updated exos -
answers: new answers made by the logged user (on another device) -
user: updates of user info or related orgs
As the server doesn't know anything about previous synchronizations, the frontend will indicate the last sync date under a from key, so the server can search only for elements created or updated after this date.
Request overview
The request MUST be a POST request on /api/sync/down with the following structure:
{
"from": 100203143,
"org_id": 3
}
-
fromis a timestamp to find elements created or updated from this date (the last sync date in the frontend) -
org_idis the org where we want to pull content (the selected org in the interface, because we don't need to pull new content for all orgs)
Response overview
Every top level field will always include 2 keys: create for new elements and update for ... you guessed it, updated elements! These keys are arrays of elements.
{
"courses": {
"create": [],
"update": [
{
"id": 3,
"description": "A fantastic course"
...
}
]
},
"skills": {
"create": [
{
"id": 443,
"name": "Overloading functions"
...
}
],
"update": []
},
...
}
Any element that was created and then updated a few minutes later, it will be only present under create.
Sync up
The frontend could sync up to the server the following elements:
- new answers
- new exos
- edited exos/changes in exos (?)
- new skills
- changes in skill
Other actions such as organization update, course creation, should only be made online because they are usually rare operations.
Request overview
The request MUST be a POST request on /api/sync/up with the following structure:
{
"answers": {
"create": [
//elements
]
},
"exos": {
"create": [
//elements
],
"update": [
//elements
]
},
...
}
All created elements MUST have an uuid field to identify them for the sync process...
All top level keys are optional, all invalid keys will be ignored. All valid but non supported keys (like courses) will get an error back.
Response overview
The server will always contain the same top level keys as given by the frontend. Top level keys contain create and/or update as array of elements created or updated. Unsupported actions (such as answers.update) with produce an error. They will always include the following:
-
success: an object as a list of UUID for elements that were successfully created. Each element is given under a sub-fieldvalue. -
fail: an object as a list of UUID for elements that caused errors. Each element contains anerrorskey as an array of errors messages. This may be an empty array.
{
//Categories list
"answers": {
"create": {
"success": {
// A list of UUID for each element success
"372d4d51-d23a-4a5b-bdb6-1a794568e308": {
"value": {
// "id": 123,
//...
},
},
"f3580b39-7fb3-4399-8848-f1e4e4879c4d": {
"value": {
// "id": 523,
//...
},
}
},
"fail": {
// A list of UUID for each element fail
"34b453bc-86a3-4cc3-8196-f675b5fca877": [
// Some errors messages:
// "Empty answer content and incorrect answer is incompatible",
// "Given exo with id 23 doesn't exist."
]
},
}
},
"exos": {
"create": {
"success": {
//idem as above
},
"fail": {
//idem as above
}
},
"update": {
"success": {
//idem as above
},
"fail": {
//idem as above
}
}
},
...
}
Answers
Request
Here is an example request with new answers:
{
"answers": {
"create": [
{
"content": {"value": false},
"correct": false,
"exo_id": 4,
"uuid": "372d4d51-d23a-4a5b-bdb6-1a794568e308",
"done_at": 5121232,
},
{
"content": null,
"correct": true,
"exo_id": 4,
"uuid": "f3580b39-7fb3-4399-8848-f1e4e4879c4d",
"done_at": 5121296,
},
{
"content": {"values": [0, 3, 4]},
"correct": false,
"exo_id": 23,
"uuid": "34b453bc-86a3-4cc3-8196-f675b5fca877",
"done_at": 5121232,
},
]
}
//... other categories
}
Answers MUST have the following properties:
-
correct: a boolean value to indicate correctness -
content: the content of the answer (as defined in db.md). Null when answer is correct. -
uuid: a UUID to identify the answer during the sync process. They are not stored in the database. -
exo_id: the associated exo ID -
done_at: the timestamp of the date when the answer has been given
TODO: can done_at be optional ??
Response
What is going to be returned by the server ? Well, it depends...
{
"answers": {
"create": [
"success": {
"372d4d51-d23a-4a5b-bdb6-1a794568e308": {
"value": {
"id": 4223,
"content": {"value": false},
"correct": false,
"exo_id": 4,
"done_at": 5121232,
"created_at": 123112332,
"updated_at": 365234221,
},
},
"f3580b39-7fb3-4399-8848-f1e4e4879c4d": {
"value": {
"id": 5112,
"content": null,
"correct": true,
"exo_id": 4,
"done_at": 5121296,
"created_at": 123112332,
"updated_at": 365234221,
},
}
},
"fail": {
"34b453bc-86a3-4cc3-8196-f675b5fca877": [
"Empty answer content and incorrect answer is incompatible",
"Given exo with id 23 doesn't exist."
]
}
]
},
...
}
TODO: return error messages TODO: return created elements to store in local state (to have the assigned ID). TODO: how to link data locally only via uuid ?? TODO: spec the creations of parent elements before child elements