Cannot get Auth Code in Oauth2 from Microsoft Azure Application

Thanks for posting - this came up somewhere else that it didn't work for all the ways that you can set up an app in active directory. It might have just been in chat. @MikeyMJCO I think?

added comp:OAuth label

added sig:bug triaged type:proposal labels

The 'offending' file is

<Drupal Root>/vendor/civicrm/civicrm-core/ext/oauth-client\providers/ms-exchange.dist.json

where the login and token links are hard coded.

I tried changing it, changing common to the tenant ID, but it didn't change what was displayed. Uninstalling and re-installing broke Oauth2 - the link redirected me back to the Admin page.

I eventually managed to get the tenant ID used. I then got a 404 error with this return.

https://<web site>/civicrm/oauth-client/return?code=some very long string&session_state=a shorter string

https://<web site>/civicrm/oauth-client/return on its own also gives Page not found.

assigned to @MikeyMJCO

Investigating this.

added 5m of time spent at 2021-04-23

I can't reproduce this on 5.36.1 on Drupal 9.1.*.

added 30m of time spent at 2021-04-23

We have the Azure AD app configured under App Registrations in Azure AD. We do have it set to "Multitenant" which is required for the "static" endpoint URL above. Is there a reason you can't set this up as a multitenant app?

You need various forms of registration for the application to be multi-tenanted, and in any case, we don't want the application to be multi-tenanted. We are just trying to set up Oauth2 authentication for IMAP and SMTP connections to our MS Exchange Online environment.

See this discussion as well: https://chat.civicrm.org/civicrm/pl/xkb6ma4x87rhtxb68hf9xfducr

@MikeyMJCO Thanks for investigating so promptly. Is the Azure application multi-tenanted? The initial error message indicated that the login needs the tenant ID in the string.

I am puzzled by why

https://<web site>/civicrm/oauth-client/return

gives 'Page not Found'. Is there some defect in my settings?

Do you get an Auth Code when you try?

Testing - it works with a Single Tenant application as well - but it needs a CiviCRM cache clear so it can get the Tenant ID.

I tried, and did a cache clear, but it didn't work with the Single Tenant application - it gave the same error message. How does it get the Tenant ID?

You can see from https://cividesktop.io/civicrm/oauth-client/return that I get an OAUTH error.

added 15m of time spent at 2021-04-23

@MikeyMJCO I think only you can see the message, as one needs to be logged in with appropriate rights to see it, but at least you are seeing an error.

We are also experiencing issues because of this. Is there a way to update the code so the urlAuthorize and urlAccessToken can use the {TenantID} instead of the hardcoded /common path in ms-exchange.dist.json?

I'm going to work up a PR for this next week.

Awesome! Thanks for the quick reply.

The approach we're going to aim for is to add a "Tenant ID" box to the Microsoft settings form, if filled in - it'll use Single-Tenant mode. If left blank it'll use current multi-tenant behaviour.

I confess to having forgotten about this issue until reminded in a StackExchange post. I have just succeeded in setting up Oauth2 authentication with MS Exchange, having returned to the fray after many months.

I edited the offending file replacing /common/ with the tenant ID.
Creating an application that uses the MS Graph API and giving it all the mail privileges.
Using the relevant mail login as appropriate.

Prompting for the tenant ID would be good.

Just confirming that "replacing /common/ with the tenant ID" is required. I'm preparing a docs PR to indicate that you need to use the MS Graph API.

I can see that the OAuthClient entity has a text/JSON field for "options" - it looks like this could be used without much code change to store urlAuthorize and urlAccessToken which would then override the defaults in ms-exchange.dist.json.

The .json file I use has these lines:

"urlAuthorize": "https://login.microsoftonline.com/tenantID/oauth2/v2.0/authorize", "urlAccessToken": "https://login.microsoftonline.com/tenantID/oauth2/v2.0/token",

where tenantID is replaced by the real one, so confirming what we need.

Does anyone have detailed docs for setting up OAuth + IMAP on Azure/Office 365? Please ping me on Mattermost (@junglebird) if you'd like to share them privately. I wrote these GMail instructions but have struggled here.

Here is what I've got - but I'm getting an error that indicates a bad password. Not sure if it's because my setup is wrong or it's because my client's Azure sign-in goes through an external SSO app.

Log into portal.azure.com
Go to Azure Active Directory (you can find it from the Search resources, invoices and docs bar at the top of the screen)
Go to Add menu » App registration.
Supported account types should be Accounts in this organizational directory only (unless you know otherwise). NOTE: Maybe we want Accounts in any organizational directory?
Redirect URI should be set to Web and the URL comes from CiviCRM (see above).
Click Register
Copy the *Application (client) ID and paste it into CiviCRM in the appropriate field.
Click Add Certificates and secrets
Click on Client secrets tab, then click the New client secret button.
Enter any description and expiration you like.
Copy the value into the Client Secret input in CiviCRM.
Go to API Permissions.
Select Add a permission.
Select Microsoft Graph.
Select Delegated Permissions.
Add the permissions listed in CiviCRM, i.e.:

https://outlook.office.com/IMAP.AccessAsUser.All
https://outlook.office.com/POP.AccessAsUser.All
https://outlook.office.com/SMTP.Send
openid
email
offline_access

I don't have access to do the Azure side but in case it's helpful I have just successfully (Finally) managed to generate a token on Azure by:

Requesting an app was set up: They sent back

ClientID:
Value:
Secret: I located the tenant id for the account and replaced common in both urlAuthorize and urlAccessToken in providers/ms-exchange.dist.json I then reinstalled the oauth extension - after making those updates, not sure if that was necessary but was indicated as being required in one of the multiple threads I found about how to get this working on Azure. I had to use the "value" that was provided not the secret- I assume that must be the id or label or something?

I found that I needed to create another file based on ../vendor/civicrm/civicrm-core/ext/oauth-client/providers/ms-exchange.dist.json where the Directory (tenant) ID replaces common in:

    "urlAuthorize": "https://login.microsoftonline.com/common/oauth2/v2.0/authorize",
    "urlAccessToken": "https://login.microsoftonline.com/common/oauth2/v2.0/token",

and keep this file somewhere else, then copy it to ../vendor/civicrm/civicrm-core/ext/oauth-client/providers/ after updates as the file gets removed. I configured the extension to use that separate file. I think you will find that ms-exchange.dist.json will get overwritten on updates, but I may of course be wrong.

Thank you @luke.stewart and @spalmstrom. I configured this app as a multi-tenant app to work around this - but I'm not sure I've configured things correctly on the Azure side. I was hoping instructions for the Azure portion existed so I can compare my changes.

As I understand it, all you would need is to allow the Directory (tenant) ID to be entered and stored as well, and use it to replace common in the relevant file. Other than that the input looks OK, I think.

You might find something in here, which includes some screenshots: #3601 (closed). The main takeaway from that though I think was don't allow User.Read since it confuses it somehow, but it doesn't look like you have set that.

I was in the same situation as Luke, where it wasn't me who eventually set up the Active Directory part, so I don't have anything to add there.

Also as a sanity check, check that IMAP is enabled on the mailbox.

Also see documentation/docs/sysadmin!335 (merged) - which is waiting for me to add screenshots

mentioned in issue #3984 (closed)

@MikeyMJCO

Did you manage to get a PR underway for this? We've bumped into this issue too, although we've managed to set things up now after customizing the tenant ID in the json file.

I will start to have a go at making the tenant ID configurable, as you proposed, but the angular parts of CiviCRM are not something I'm very familiar with, so if you've already made a start that would be great [=

unassigned @MikeyMJCO

No progress on a PR and no longer any real active CiviCRM usage to justify expending the effort.

I ran into this issue as well and used the steps outlined above. I suspect there are more user who have simply implemented the (not ideal) workaround but haven't commented on it.

Feel free to assign this to me, I'll put a PR up for a few in a couple of days [=

OK, I've pushed up a PR: https://github.com/civicrm/civicrm-core/pull/25214

Which:

Supports a new option in provider definitions for 'tenancy'
Adds the tenant ID to the oauth client table
Displays a UI option to add a tenant ID when the provider's 'tenancy' option is TRUE
Adds tenancy support and {{tenant}} token to the OAuth provider URLs for ms-exchange
~~New provider class to handle ms-exchange~~, Update civigenericprovider class with a helper function to rewrite the {{tenancy}} part of the URL with a tenancy ID where it is present, otherwise default it to 'common'

Not sure if this is all the best way to do it, but it felt like a re-usable approach, if it needed to be implemented in a similar way for GMail/Google Workspaces or other providers.

Very happy to take any feedback. But if it's good then I will:

~~Add tests~~ Done
Update the documentation

added has-pull-request label

mentioned in commit 8e454e9a

mentioned in commit 88b3982f

changed milestone to %5.59.0

Resolved by @JKingsnorth

closed

Thanks @mattwire - can you reopen to remind me to do the docs PR?

reopened

added needs:docs need update label

@JKingsnorth reopened

mentioned in commit JKingsnorth/dev@802b2ae5

mentioned in merge request documentation/docs/dev!1052 (merged)

mentioned in commit JKingsnorth/sysadmin@5b785d35

mentioned in merge request documentation/docs/sysadmin!346 (merged)

@mattwire docs PRs, would appreciate some sense-checking on whether you think this is understandable and sufficient?

Developer documentation update: documentation/docs/dev!1052 (merged)

Sysadmin guide: documentation/docs/sysadmin!346 (merged)

mentioned in commit documentation/docs/dev@1d0e5b2c

mentioned in commit documentation/docs/sysadmin@2593ac05

Cool, thanks @mattwire , I guess this can close now [=

Hi,

Thank you for raising an issue to help improve CiviCRM. As you may know, this issue has not had any activity for quite some time, so we have closed it.

We would like you to help us to determine if this issue should be re-opened:

If this issue was reporting a bug, can you attempt to reproduce it on a latest version of CiviCRM?
If this issue was proposing a new feature, can you verify if the feature proposal is still relevant? Did it get the concept-approved label? Have other people also shown interest? Could it be implemented as an extension?

If the answer to either question is yes, please feel free to comment or re-open the issue. Please also consider:

Is it something that you could help implement, either by sending a patch or hiring someone who can?
Would you be willing to fund the work, either through a CiviCRM partner, or the Core Team?

Thank you for your help and contributions to CiviCRM.

P.S. This is an automated message, see infra/gitlab issue 20. We understand that automatic responses are annoying, but given the number of open issues as the project evolves, we need a bit of help to triage and prioritise the most relevant issues.

closed

Cannot get Auth Code in Oauth2 from Microsoft Azure Application

Overview

Reproduction steps

Current behaviour

Expected behaviour

Environment information

Comments

Child items ...

Activity