[AIT-236] Add partial object sync specification for protocol version 6+#413
[AIT-236] Add partial object sync specification for protocol version 6+#413JoaoDiasAbly wants to merge 3 commits intomainfrom
Conversation
|
Once the protocol is agreed it would be good if either @VeskeR or I could be added as a reviewer here too, since we'll be the ones implementing it 🙂 |
thanks for the heads-up! I will add you straight away if that's alright 🙏 |
|
Gonna add myself too for visibility |
See spec ably/specification#413 Resolves AIT-38
textile/objects-features.textile
Outdated
| *** @(RTO5a5)@ An @OBJECT_SYNC@ may also be sent with no @channelSerial@ attribute. In this case, the sync data is entirely contained within the @ProtocolMessage@ | ||
| ** @(RTO5b)@ During the sync sequence, the "@ObjectMessage.object@":../features#TR4r values from incoming @OBJECT_SYNC@ @ProtocolMessages@ must be temporarily stored in the internal @SyncObjectsPool@ list | ||
| ** @(RTO5b)@ During the sync sequence, the "@ObjectMessage.object@":../features#TR4r values from incoming @OBJECT_SYNC@ @ProtocolMessages@ must be temporarily stored in the internal @SyncObjectsPool@ list. The behavior differs based on the protocol version in use: | ||
| *** @(RTO5b1)@ For protocol versions prior to 6, the server sends a complete @ObjectState@ for each object in a single @OBJECT_SYNC@ @ProtocolMessage@. Each @ObjectState@ received is stored directly in the @SyncObjectsPool@ |
There was a problem hiding this comment.
The spec describes the behaviour for a single protocol version (that given by CSV2b); we don't need to maintain the description of the v5 behaviour.
textile/objects-features.textile
Outdated
| ** @(RTO5b)@ During the sync sequence, the "@ObjectMessage.object@":../features#TR4r values from incoming @OBJECT_SYNC@ @ProtocolMessages@ must be temporarily stored in the internal @SyncObjectsPool@ list. The behavior differs based on the protocol version in use: | ||
| *** @(RTO5b1)@ For protocol versions prior to 6, the server sends a complete @ObjectState@ for each object in a single @OBJECT_SYNC@ @ProtocolMessage@. Each @ObjectState@ received is stored directly in the @SyncObjectsPool@ | ||
| *** @(RTO5b2)@ From protocol version 6 onwards, partial object sync is enabled. The server may split a large object across multiple @OBJECT_SYNC@ @ProtocolMessages@, each containing a partial @ObjectState@ for the same object. Multiple @OBJECT_SYNC@ messages may be delivered with the exact same @channelSerial@ when they contain partial data for the same object. The client determines that sync messages relate to the same object by matching the @ObjectState.objectId@. When receiving partial @ObjectState@ messages for the same @objectId@: | ||
| **** @(RTO5b2a)@ If an @ObjectState@ with the given @objectId@ does not yet exist in the @SyncObjectsPool@, store the received @ObjectState@ |
There was a problem hiding this comment.
I don't think that it makes sense to, in RTO5b2, talk about "when receiving partial ObjectState messages for the same objectId", and then in RTO5b2a describe what to do if that objectId doesn't exist yet; the child is not a sub-scenario of its parent.
textile/objects-features.textile
Outdated
| ** @(RTO5b)@ During the sync sequence, the "@ObjectMessage.object@":../features#TR4r values from incoming @OBJECT_SYNC@ @ProtocolMessages@ must be temporarily stored in the internal @SyncObjectsPool@ list | ||
| ** @(RTO5b)@ During the sync sequence, the "@ObjectMessage.object@":../features#TR4r values from incoming @OBJECT_SYNC@ @ProtocolMessages@ must be temporarily stored in the internal @SyncObjectsPool@ list. The behavior differs based on the protocol version in use: | ||
| *** @(RTO5b1)@ For protocol versions prior to 6, the server sends a complete @ObjectState@ for each object in a single @OBJECT_SYNC@ @ProtocolMessage@. Each @ObjectState@ received is stored directly in the @SyncObjectsPool@ | ||
| *** @(RTO5b2)@ From protocol version 6 onwards, partial object sync is enabled. The server may split a large object across multiple @OBJECT_SYNC@ @ProtocolMessages@, each containing a partial @ObjectState@ for the same object. Multiple @OBJECT_SYNC@ messages may be delivered with the exact same @channelSerial@ when they contain partial data for the same object. The client determines that sync messages relate to the same object by matching the @ObjectState.objectId@. When receiving partial @ObjectState@ messages for the same @objectId@: |
There was a problem hiding this comment.
Multiple @OBJECT_SYNC@ messages may be delivered with the exact same @channelSerial@ when they contain partial data for the same object.
I think that this is a detail that clients don't need to concern themselves with, and which may confuse rather than enlighten (because it becomes unclear whether they're meant to do something as a result of receiving duplicate channelSerials). All they care about are two things:
- is this a new sync sequence?
- is this the end of the sync sequence?
— both of which are already addressed in the existing spec
textile/objects-features.textile
Outdated
| *** @(RTO5b2)@ From protocol version 6 onwards, partial object sync is enabled. The server may split a large object across multiple @OBJECT_SYNC@ @ProtocolMessages@, each containing a partial @ObjectState@ for the same object. Multiple @OBJECT_SYNC@ messages may be delivered with the exact same @channelSerial@ when they contain partial data for the same object. The client determines that sync messages relate to the same object by matching the @ObjectState.objectId@. When receiving partial @ObjectState@ messages for the same @objectId@: | ||
| **** @(RTO5b2a)@ If an @ObjectState@ with the given @objectId@ does not yet exist in the @SyncObjectsPool@, store the received @ObjectState@ | ||
| **** @(RTO5b2b)@ If an @ObjectState@ with the given @objectId@ already exists in the @SyncObjectsPool@, merge the entries from the received @ObjectState@ into the existing one: | ||
| ***** @(RTO5b2b1)@ For a map object, merge @ObjectState.map.entries@ from the received @ObjectState@ into the existing @ObjectState.map.entries@. Other fields on the @ObjectState@ envelope (such as @siteTimeserials@) and the map envelope (such as @semantics@) will be identical across all partial messages for the same object; the client may apply these from any message relating to that object |
There was a problem hiding this comment.
For a map object
to be determined how, from the given ObjectState? presumably by checking whether ObjectState.map is present
textile/objects-features.textile
Outdated
| *** @(RTO5b2)@ From protocol version 6 onwards, partial object sync is enabled. The server may split a large object across multiple @OBJECT_SYNC@ @ProtocolMessages@, each containing a partial @ObjectState@ for the same object. Multiple @OBJECT_SYNC@ messages may be delivered with the exact same @channelSerial@ when they contain partial data for the same object. The client determines that sync messages relate to the same object by matching the @ObjectState.objectId@. When receiving partial @ObjectState@ messages for the same @objectId@: | ||
| **** @(RTO5b2a)@ If an @ObjectState@ with the given @objectId@ does not yet exist in the @SyncObjectsPool@, store the received @ObjectState@ | ||
| **** @(RTO5b2b)@ If an @ObjectState@ with the given @objectId@ already exists in the @SyncObjectsPool@, merge the entries from the received @ObjectState@ into the existing one: | ||
| ***** @(RTO5b2b1)@ For a map object, merge @ObjectState.map.entries@ from the received @ObjectState@ into the existing @ObjectState.map.entries@. Other fields on the @ObjectState@ envelope (such as @siteTimeserials@) and the map envelope (such as @semantics@) will be identical across all partial messages for the same object; the client may apply these from any message relating to that object |
There was a problem hiding this comment.
We also need to describe which of the various ObjectMessages' serialTimestamp properties should be used if we need to tombstone the object when we apply the sync objects pool.
There was a problem hiding this comment.
If a tombstoned object is received over a sync sequence, the realtime guarantees the invariant that there will only ever be a single ObjectMessage for such an object.
The reason is that 1. tombstoned objects can't be un-tombstoned - that operation is terminal - and 2. an object's state is cleared upon tombstoning, which means there is nothing to paginate across multiple messages during a partial sync sequence, as the effective size of a tombstoned object is 0.
I think I'll add an explicit spec point for serialTimestamp for tombstoned objects with a small note.
See slack discussion
There was a problem hiding this comment.
If a tombstoned object is received over a sync sequence, the realtime guarantees the invariant that there will only ever be a single ObjectMessage for such an object.
I don't think we need to add anything explicit to the spec regarding handling serialTimestamp during a partial sync sequence. There is only an ObjectMessage for a tombstoned object, which we will set in the sync pool, and then spec points RTLC6f, RTLM6f and RTLO4e use the correct serialTimestamp as a result.
I did decide to add a failsafe check for when we receive a second ObjectMessage for the same objectId that has a tombstone flag set, though. Based on the DR client implementation steps, "The tombstone flag always wins if set", which essentially means just use the ObjectMessage that has the tombstone flag set in its entirety. See RTO5f2a1 in f6722ad
VeskeR
changed the title
See spec ably/specification#413 Resolves AIT-38
VeskeR
force-pushed
the
feature/AIT-236
branch
from
b23c73e to
444a2d3
Compare
See spec ably/specification#413 and a DR [1] Resolves AIT-38 [1] https://ably.atlassian.net/wiki/x/AQBxCQE
See spec ably/specification#413 and a DR [1] Resolves AIT-38 [1] https://ably.atlassian.net/wiki/x/AQBxCQE
3 participants
Ticket: https://ably.atlassian.net/browse/AIT-236
Follow-up from: https://ably.atlassian.net/browse/AIT-127