I am downloading real dataset (and already downloaded fake from huggingface). To download Laion subset, I am using: simple_laion400m_elsa_d3_subset_download.py. I followed the instructions and script runs successfully.
However, it seems the script gets stuck at download() for training data and neither it throws error nor does it run further commands. I tried changing process counts, number of samples per shard to make the process less RAM heavy. Nothing seems to work. Is there a better way to download Laion subset?
I am downloading real dataset (and already downloaded fake from huggingface). To download Laion subset, I am using: simple_laion400m_elsa_d3_subset_download.py. I followed the instructions and script runs successfully.
However, it seems the script gets stuck at download() for training data and neither it throws error nor does it run further commands. I tried changing process counts, number of samples per shard to make the process less RAM heavy. Nothing seems to work. Is there a better way to download Laion subset?