There are two possible solutions:
1- Create a global object, which you use to save the IDs for all entries, you could check and save the ID for each entry before pushing it to the dataset.
2- You could use this Actor to remove duplicates, by running it on the dataset of your Actor after it finishes:
https://apify.com/lukaskrivka/dedup-datasets