Hi, on Apify platform, you can store your data either to
dataset or
key value store. You can
store any valid JSON in a dataset, so you can structure your data any way you want. Does this answer you question?
Regarding elimination of duplicates, we usually keep track of IDs (e.g.
HashSet) of items that we have already pushed to dataset. Everytime we're pushing a new item, we check whether it's already been pushed.
