Skip to content
Snippets Groups Projects
New look: Off

updateAllMemberships in memberschip BAO crasches out of memory on large numbers of members

    • Closed Issue created by Joost @Joost

    We have a database with about 1.3 million memberships. the scheduled task to update membership statuses crashes with an out of memory error.

    To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information

    Child items ...

    • Activity

    • eileen added needs:concept approval triaged type:proposal labels

      added needs:concept approval triaged type:proposal labels

    • eileen
      eileen @eileen ·
      Owner

      Hmm - I doubt the job was written with that volume of memberships in mind!

      You may find this patch helps https://github.com/civicrm/civicrm-core/pull/12276

      Otherwise it's probably worth writing a new function in an extension (that could eventually replace the core one)

    • Joost
      Joost @Joost ·
      Author

      I will allready try to lessen the memory impact of the function by not getting a dao for all memberships, but by extending the query before creating a DAO.

    • eileen
      eileen @eileen ·
      Owner

      Ah - found this - we never got it successfully merged & Johan changed role = but it might help - https://issues.civicrm.org/jira/browse/CRM-19920

    • eileen
      eileen @eileen ·
      Owner

      Actually - maybe looking at git blame we DID merge it

    • eileen
      eileen @eileen ·
      Owner

      (or perhaps only part)

    • eileen
      eileen @eileen ·
      Owner

      (note the memory leak patch above is still worth trying )

    • Joost
      Joost @Joost ·
      Author

      I was allready looking at that one (I took over Johan's function), however as you said it isn't easy to merge.

    • ****
      **** @support_bot_dev ·
      Reporter

      Hi,

      Thank you for raising an issue to help improve CiviCRM. We're sorry about this, but this particular issue has gone unnoticed for quite some time.

      We would like to ask you to help us out and determine whether this issue should be kept open:

      • If this issue is reporting a bug, can you attempt to reproduce on the latest version of CiviCRM?
      • If this issue is proposing a new feature, can you verify whether the feature proposal is still relevant? Did it get the concept-approved label? Have other people also shown interest? Could it be implemented as an extension?

      If yes,

      • Is it something that you could help implement, either by sending a patch or hiring someone who can?
      • Would you be willing to fund the work, either through a CiviCRM partner, or the Core Team?

      Please feel free to comment or re-open the issue. We understand that automatic responses are annoying, but as the project evolves, we need a bit of help to triage and prioritise the most relevant issues.

      Thank you for your help

      P.S. This is an automated message. See infra/gitlab issue 20.

    • **** closed

      closed

    Please register or sign in to reply