Caching of CustomField metadata is a mess
Background
Multiple forms and functions need a list of custom fields. For example:
- Import forms need to know which custom fields can be imported
- SearchKit needs to know what fields can be searched & displayed
- Every crud form needs to display custom data sets
- Afform needs the list of custom fields
- etc. etc. etc.
Due to time, entropy and other "real world" stuff, most of these have developed their own peculiar way of gathering the list of custom fields, and implemented their own quirky ways of caching it.
What it should look like
A sensible model would be to have 1 function that returns a cached list of all custom fields. Caching would happen only once, and only in this function. All other code that needed to fetch a list of custom fields would call this function. All transformations of the master list (e.g. filtering by entity type, filtering by user permissions) should happen downstream and should not be re-cached.
What it actually looks like
The technical term for what we have now is "a mess." No one function exists. Lots of redundancy. Some places don't cache at all which kills performance (APIv4 getfields included). Some other places create huge bloated caches which waste memory. Yet other places take those already cached lists and waste even more memory by caching them again!