Standalone: users and roles

General issue to discuss user management, roles and permissions.

There was a discussion in Manchester about this. At first we explored the idea of using CiviCRM groups to manage permissions, but there was a lot of discomfort because of the (lack of) security around groups and how it could end up adding a lot of extra complexity. Of course, maybe an implement or another might prove one way or another.

So far one WIP branch by @DaveD proposes creating civicrm_user with an ID, username, password and maybe email. While testing, I managed to get it working by also adding a record in the civicrm_uf_match table (for authx http logins), to link that user to a contact.

Presumably we would also have civicrm_user_role (ex: admin, staff, member) and civicrm_user_permission (ex: "admin" has the "Administer CiviCRM" permission).

And then we would have the same permission grid similar to what CiviCRM has for WordPress role management (in that case, it adds WordPress capabilities, but in this case, it would add records in civicrm_user_permission).

cc @DaveD @artfulrobot @JoeMurray

Related meta: #2998 (closed)

added sig:standalone label

mentioned in issue #2998 (closed)

marked this issue as related to #2998 (closed)

added triaged label

I'm in favour of using new tables, rather than trying to make groups do something new (especially as groups are not used for user management on any other frameworks).

I wonder what the process would look like when migrating from a framework to standalone, or vice-versa. Would the new standalone tables exist on all installs, or would there be some sort of setup process when setting up a standalone installation?

CiviCRM groups are currently used for ACLs, but I think most people find ACLs complex enough.

It's not always obvious to explain why some permissions are in the CMS, some are in ACLs. To recap, CMS permissions are like "gate-keepers" (or fences), and ACLs let you decide what you can view/edit once you are past the gate.

Still, that means CiviCRM has two basic permission systems, and it makes authenticating difficult when doing things such as API or checksum access. If I recall correctly, we can't give API access without a CMS password-based account, even if the password is never used.

On the other hand, I think it's better to avoid adding a special behaviour for Standalone, and make it work similar to all other integrations, both for making it easy for developers to adopt it, and to avoid situations of "if you're on X, then you can use Y" (fragmenting the community).

Good question regarding the tables. I would leave the question of the database tables up to the implementation. The WIP branch by DaveD generates the DAOs only if the 'CMS' (UF) is Standalone, and maybe we don't want to expose the DOAs on other UFs. So if it's the case, then we need some easy method for making sure the tables are created.

Example migration steps:

• Install CiviCRM Standalone in a new folder, run the CiviCRM installation
• Dump and import the CiviCRM tables from the old Drupal/WordPress/Joomla installation into the CiviCRM Standalone database
• Run a user migration tool, which would probably need the MySQL credentials of the old installation. We could have extensions that provide migrations for each CMS.
• Setup the roles/permissions from scratch (maybe migration-extensions could do it, but the cost/benefit isn't great).

Similar to the Drupal to WordPress migration:
https://docs.civicrm.org/sysadmin/en/latest/misc/switching-cms/drupal-to-wordpress/

I think the main caveat is that the two installations (the old CMS one, and CiviCRM Standalone) should be fairly close in versions, to avoid major schema changes. The upgrader will re-run and fix any differences, but it does not always work.

I think it is an elegant approach to use a new table for standalone to mimic the functionality provided by Drupal and WordPress user permissions. I really like having id, username, password and email associated with a user and using the civicrm_uf_match table for standalone; more fields beyond that are not necessary, not problematic, but potentially not a shared use case for all standalone. Ensuring that an authx based login works nicely, including by ensuring there is an entry in civicrm_uf_match like for all CMS integrations, seems wise.

I agree - I think we can move forward confidently in agreement - that we don't want to use groups/ACLs for this.

I started on this last year, too, with the following new entities:

• User (civicrm_user)
• Role
• RolePermission
• UserRole

My intention was to do as Drupal (7, though think it's the same in 8+) does. I think there's some useful/important security stuff in there, especially an email address for the users separate to the email table.

I was intending to get this working as an extension, as I'm completely new to tinkering with install scripts etc.

I'm not sure about WP (it's been ages), but I liked the following schema:
https://github.com/artfulrobot/civicrm-core/tree/artfulrobot-standaloneusers/ext/standaloneusers/xml/schema/CRM/Standaloneusers

I think I got stuck trying to get SK to do something (resulting in Coleman's replacement of the entity ref UI in 5.57!) and then ran out of free time. There's not much to look at other than the schema.

There's a couple of other commits I needed to get standalone working ok in there which belong to https://github.com/civicrm/civicrm-core/pull/24955

Wonder if anyone thinks using Symfony's libs might be sensible here, given that Civi is a Symfony app? Or might this confuse stuff further? https://symfony.com/doc/current/security.html

The "firewall" bit seems to overlap Authx, and it's not clear if the ORM is required?

But if there are things that are easily usable, sounds interesting.

(I find the Symfony docs rather difficult, from the point of view of a civicrm dev, because it's not really a Symfony app)

@bgm yes I agree re Symfony docs.

I like the idea (which, incidentally, I should give credit to @nicol for) to use well tested library code for auth stuff, but I have struggled to find anything suitable. For example, looking at the list of most starred authorization projects in PHP on Github https://github.com/topics/authorization?l=php

• oauth2 - no
• bouncer - laravel
• huge - has reached "soft end of life"
• laratrust - laravel
• sentinel - laravel
• php-casbin - incomplete for our needs.
• php-auth maybe
• 2 more laravel ones
• openid... - been there, no thanks.
• yet more laravel ones...

Given how many moving parts there are with CMS authentication: registration, login, session handling, password reset, username reminder, email verification, logout, throttling, changing details, roles, user role admin, session logging, 2FA, potentially even GDPR handling (consent, data access requests, data deletion) – it does seem appealing to hand this over to a third party library with a bigger group of users/testers/security professionals than Civi users.

Figuring out Symfony or PHP-Auth or something else (SoLiD lol) sounds an up-front headache, but perhaps saves time/pain later (while also making it easier for an org to risk assess the long-term consequences of switching to standalone).

It's a good point. On the other hand, as long as Civi has to also live on top of RBAC CMSes like WP, Drupal, Joomla we can't go too off-grid here. I don't think Solid (I'm assuming you mean the decentralisation project but please explain if not), applies because it's decentralised and civi is centralised; we want an admin to be able to configure what all users are able to do. While some decentralised information store could potentially be linked to for some bits n pieces maybe, I don't see its relevance to who to let into the CRM - this could be my ignorance speaking.

Also, there are problems with using 3rd party libs, because when they change we have to. Oftentimes Civi's proved itself not that good at floating as tides change - e.g. AngularJS, Smarty 2, jQueryUI all come to mind where the projects have moved on and we haven't. So combine this with the challenge above of "it must use the same pattern as our regular CMSes" and you're introducing a risk of brittleness while also adding more dependencies.

I didn't get a notification of this reply so just saw it…

SoLiD - I added a lol to try to indicate it wasn't a serious suggestion.

3rd party libs - I hear you. The flip side is Civi also has a long (proud?) history of unique hacks to do things. Something like authentication seems like it benefits from using a framework with other people keeping an eye on it as well. I'm not sure Symfony and Smarty are sooo comparable..

Quick update after the mini sprint on 3 Feb.

• we're using standard Civi entities to provide the necessary tables for traditional RBAC (role based authentication control) that CMSes provide.
• we've identified the list of work to get this done, and started on it
• We're trying to keep PRs to core things as small as possible to ease merging, and to do as much as possible in a new extension. Currently there's a lot in the core fork that can probably be moved to the extension.

The extension is at https://lab.civicrm.org/extensions/standaloneusers/

(its key is standaloneusers but it actually has broader remit than that, it's just that it's hard to rename an extension once it's built and it's not seen as a priority)

I (@artfulrobot) currently have a repeatable way to install standalone using composer, and is working on the buildkit install. But currently this means that most of the above repos are forked, if only for the purposes of bringing in my forked versions of the other repos! It's a right mess to work on because of this.

Here's a summary of what is forked and why

• civicrm-buildkit
  • My fork https://github.com/artfulrobot/civicrm-buildkit/tree/artfulrobot-standalone brings in my fork of cv and also has/will have necessary changes that will result in a PR/MR.
• civicrm-standalone
  • My fork of this brings in the artfulrobot-standalone branch of my fork of core https://github.com/civicrm/civicrm-standalone/compare/master...artfulrobot:civicrm-standalone:master
• cv
  • My fork https://github.com/civicrm/cv/compare/master...artfulrobot:cv:artfulrobot-standalone teaches cv how to identify and boot standalone. This will need merging, once I've got it working.
• civicrm-core
  • My fork https://github.com/civicrm/civicrm-core/compare/master...artfulrobot:civicrm-core:artfulrobot-standalone implements various standalone classes, and includes work on authx. This will need a merge once stable.
• a new extension (see below) that will one day be a core ext but is only needed for standalone.

I'm aware that a lot of this has my name on it. If you're interested in working on this with me (fab!) then we can talk to the relevant people and try to get something a bit less artfulrobot-y set up and/or I can make you co-owners of the above etc.

Those of you who know me might know that I love a diagram. So here's one I drew to help me get my head around how all these fit together.

A few weeks ago, we did some revisions+merges on a few parts. Notably:

• cv.git: Should work with mainline/current.
• civicrm-buildkit.git: Should work with mainline/current. To use a specific fork/branch, pass --cms-ver:

  civibuild create st --type standalone-clean \
    --cms-ver artfulrobot/artfulrobot-clean-install-paths
  
  civibuild create st --type standalone-clean \
    --cms-ver totten/migrated

One still needs to pull code from at least three places (civicrm-standalone.git, civicrm-core.git, standaloneusers.git). Of course, we want to simplify (having it spread-out increases mental-load for installation, development, patch-tracking, etc). I've also chatted with a couple folks (Eileen, Seamus, Coleman) to get more thoughtlets on the approach to repo-layout/versioning/merging/etc.

Things which stood out:

• Standalone is generally in a development/incubation period -- i.e. it's not very important to think about backward-compatibility/migration. It's more important to get changes integrated with low-overhead.
• We should clarify the version-numbering -- so that it's easier to verbalize about bugs/improvements/etc.

@artfulrobot I'd like to propose this general policy for the foreseeable future -- the incubation policy:

• civicrm-standalone.git holds all code that is distinctive to the standalone deployments -- Standalone.civi-setup.php, CRM/Utils/System/Standalaone.php, ext/standaloneusers, and so on. As a user/tester/developer/contributor/etc, you should be able to get everything by git clone civicrm-standalone.git.
• If you like a patch, then just push it. PR's can be used if one actively wants a discussion (e.g. because the change feels risky/big; e.g. because it's a topic the author hasn't worked on before).
• Versions will be branched/tagged pro-forma on a monthly basis (in parallel with civicrm-{core,drupal,...}). This doesn't mean that it's expected to work. (The repo is still in incubation.) But it gives a clear reference-point for reporting bugs, testing compatibility (core.git<=>standalone.git), and so on.

(It's alternatively possible to do a fully integrated incubation policy -- where everything except a composer.json lives in civicrm-core.git and where we have an understanding that "Standalone" patches are more relaxed. There are some advantages and some precedent among core-exts, and I'd also be OK with this. But I think it may be easier for more people to understand if the scope of incubation-policy matches the scope of civicrm-standalone.git.)

The incubation policy would probably stay in place for some #months -- until we have a critical-mass of user/role functionality, automated testing, etc.

↑ just to update that.

We've moved on a bit and moved quite a bit of code around. See https://github.com/artfulrobot/civicrm-standalone/ for up-to-date summary.

mentioned in issue #4466 (closed)

There's a related question: Once you have the ability to define roles and permissions... What roles should we define by default? How do we maintain those roles? Split off a separate issue for that question: #4466 (closed)

We have users and roles now. The UX is a bit rough, but mostly functional; you can create roles with certain permissions and apply those roles to users, and they're honoured.

I'm going to close this as it's a general topic, and I think it would be more helpful at this stage to have focused issues on single things rather than big topics.

closed