Add tracking table for import jobs
A current blocker to refactoring the import jobs to use queue processes / non-timeout UI methods is persistence of output data.
We discussed this in Barcelona and came up with the following proposal
- create a new table civicrm_user_job with the fields
- contact_id (or created_id?)
- job_identifier (or name?) - we probably would need this to be unique but the BAO could handle appending to it if it is not.
- start_timestamp
- end_timestamp
- job_hash - used for file naming
-
files created as a result of the job could have a standard naming convention - e.g import_{job_hash}validation_errors.csv import{job_hash}_duplicates.csv
-
We would need a cleanup job which would also remove the files - e.g more than one week old
-
The UserJob BAO would obviously run pre & post hooks - allowing tracking by extensions
-
Permissioning of file retrieval can be via Attachment api - Tim thinks we know how that works but he is wrong.
-
When viewing the results of the import this creates options for us to potentially present a user with their imports (plural) results & for them to choose which they want. (Perhaps exposing the job_idenfier field to them would help here?