Skip to content
GitLab
  • Menu
Projects Groups Snippets
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • CiviCRM Core CiviCRM Core
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 1,439
    • Issues 1,439
    • List
    • Boards
    • Service Desk
    • Milestones
  • Deployments
    • Deployments
    • Releases
  • Wiki
    • Wiki
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
Collapse sidebar
  • Development
  • CiviCRM CoreCiviCRM Core
  • Issues
  • #1307
Closed
Open
Created Oct 09, 2019 by eileen@eileen🎱Owner

Add tracking table for import jobs

A current blocker to refactoring the import jobs to use queue processes / non-timeout UI methods is persistence of output data.

We discussed this in Barcelona and came up with the following proposal

  1. create a new table civicrm_user_job with the fields
  • contact_id (or created_id?)
  • job_identifier (or name?) - we probably would need this to be unique but the BAO could handle appending to it if it is not.
  • start_timestamp
  • end_timestamp
  • job_hash - used for file naming
  1. files created as a result of the job could have a standard naming convention - e.g import_{job_hash}validation_errors.csv import{job_hash}_duplicates.csv

  2. We would need a cleanup job which would also remove the files - e.g more than one week old

  3. The UserJob BAO would obviously run pre & post hooks - allowing tracking by extensions

  4. Permissioning of file retrieval can be via Attachment api - Tim thinks we know how that works but he is wrong.

  5. When viewing the results of the import this creates options for us to potentially present a user with their imports (plural) results & for them to choose which they want. (Perhaps exposing the job_idenfier field to them would help here?

@totten @pfigel @seamuslee

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
Time tracking