readhelper

func FilterSortedRecords ¶

func FilterSortedRecords(data *ajson.Node, recordsKey string, since time.Time,
	timestampKey string, timestampFormat string, nextPageFunc common.NextPageFunc,
) ([]map[string]any, string, error)

FilterSortedRecords filters and returns only the records that have changed since the last sync, based on a provided timestamp key and reference value.

Records has to be sorted, recently updated first.

It compares each record's timestamp (identified by the `timestampKey`) against the provided `since` time. Only records with timestamps greater than the given `since` time are considered new or updated and included in the result.

Parameters:

• data: A JSON node containing an array of records to filter
• recordsKey: The JSON path to the array of records within the data node
• since: The reference timestamp; only records newer than this will be returned
• timestampKey: The field name within each record that contains the timestamp to compare
• timestampFormat: The time format string used to parse timestamps from the provider
• nextPageFunc: Function to generate next page token if more records are available

Returns:

• A slice of maps containing only the records updated after the specified timestamp
• Next page token string if more records are available, empty string if reached end
• An error if parsing timestamps, comparing values, or processing fails

Example:

updatedRecords, nextPage, err := common.IncrementalSync(
    data,
    "records",
    lastSyncTime,
    "updated_at",
    time.RFC3339,
    nextPageFunc,
)