Update last seen




  • Official comment
    Geir Ove Grønmo

    You don't mention which source type you have in mind. The merge source is technically a black box and cannot be rewound. The merge_datasets and union_datasets sources uses a dot notation in their _updated values. You can edit those manually if you want.

    What is the use-case for rewinding a single source?

    Comment actions Permalink
  • erik.leven

    Yeah, this was regarding the merge source. Well, when I do set the "update last seen"-value through dot separation, starting with the first dataset in the datasets poperty of the source, it very often gives me some weird results. I'll try to show an example.

    In this case I'm merging 4 datasets. The datasets property of the source is the following:

    "datasets": ["cvpartner-customer cvc", "currenttime-company cc", "currenttime-company-extra ccx", "ad-customer ad"]

    Starting with cv-partner-customer and ending with ad-customer, their most recent _update-values are:


    If I want to re-send only entities from currenttime-company, from _updated value 5000 I would expect to write 629.5000.6044.44407 in the "update last seen" option.

    This is the result:


    In other words, nothing happens after I hit refresh. I often find it hard to correctly do this for merge sources, and quite often I either get no result or the source resend entities I did not specify. That's why I was wondering if there could be a way to specify one dataset individually, instead of marking down all of the, This can also be quite tedious when there are 20 datasets feeding into the source.

    Comment actions Permalink
  • Geir Ove Grønmo

    Do you mean "merge_datasets"? The "merge" source does not have _updated values like 629.5105.6044.44407.

    Comment actions Permalink
  • Geir Ove Grønmo

    The "merge" source does not support rewinding; well technically it does, but it makes no attempt to change the merging part itself. It just means that the *already merge* entities will come out of the source once more and run through the transforms and into the sink. You cannot use rewinding to reapply equality rules. Only resetting the pipe will do that.

    The "merge_datasets" source does support rewinding like the one you showed (where you rewind one of the offsets in the composite since value).

    Comment actions Permalink

Please sign in to leave a comment.