To complete our current article on how to setup a CRON task to automate CSV imports, you can also configure multiple CRON tasks to import large CSV files multiple times.
To do so:
- In the CRON php file, configure the option 'olderthan' to 120 minutes for example. This way files that have been modified less than 120 minutes ago will not be processed.
- The option "deletetodofile' should be set to '0' allowing the CRON tasks to continue the same import instead of starting from the beginning of your original file.
- Within the Import panel in Store Commander (top panel), modify the number of lines to import by a quarter of the total lines existing in your CSV file. For instance, if the file contains 1956 lines, configure the option 'number of lines' to 500 for 4 CRON tasks (500*4=2000 lines to be processed in total).
- Then run the CRON tasks multiple times in a row (in our example: 4 times every 20 minutes. You need to keep a 20 minutes interval to avoid importing the same data multiple times).
This way, your large CSV files will be processed in 4 parts rather than one. You can of course modify the above settings to suit your needs, depending on the size of the CSV files.
Download our themed Guides to learn how to manage your shop at lightning speed with Store Commander.