Less than 1 week –
Less than 10 hrs/week –
We have a list of larger CVS files, which daily are to be imported into a Elasticsearch cluster.
The CSV files vary in size, but the job consists of about 300-500 files with about 3-4 million rows in total.
All files in one job are to be imported into same index. However, up to 10 jobs are to run daily, result in a daily update of about 30-40 million records.
Today we are able to perform the task with single ...