Resolving BullPhish ID email deliverability conflicts in Microsoft 365

Introduction

BullPhish ID uses specific domains and IP addresses to securely send phishing and training campaign emails to your end users. By applying relevant safelisting protocols, BullPhish ID is configured to ensure email deliverability to campaign end users. However, certain SaaS-based B2B email security platforms or email server configurations can cause conflicts for users of Microsoft 365.

The schema below illustrates how delivery of email messages to Outlook 365 customers can involve communications between many layers of customer infrastructure. The complexity of this infrastructure could cause various conflicts affecting the end user experience, including delivery and tracking issues that may result in BullPhish ID phishing and training campaign messages not being delivered to intended end users.

This guide will help you identify and resolve the root causes of most common email deliverability conflicts for Microsoft 365 users.

NOTE  Tracking issues will be addressed in a separate article.

Prerequisites

• You must be a member of the Security Administrator role group in the Microsoft Security & Compliance Center.

• You must be a member of the Organization Management role group in Microsoft Exchange Online.
• For more information about Microsoft permissions, see the Microsoft articles Microsoft Defender for Office 365 permissions in the Microsoft Defender portal and Permissions in Exchange Online.

Identifying email deliverability issues

The steps to troubleshooting email deliverability issues begin with identifying the email delivery environment.

1. If the end user receives an email message in their inbox that contains a security banner, skip to step 10.
2. If the end user does not receive an email message, check the status of the campaign and email message from within BullPhish ID.
3. Log into BullPhish ID and navigate to the Training & Awareness Dashboard or the Phishing Dashboard. All steps in identifying email deliverability issues apply to both dashboards. There you will find a list of campaigns. In the Status column, check to ensure that the current campaign status is not listed as Scheduled or Cancelled.
   
   On the campaign's Details page, check that the campaign start date and time are in the past.
   

NOTE  Campaigns are sometimes cancelled automatically because of issues with the account subscription or the availability of content associated with campaigns.

4. Open the campaign Details page and hover over the i icon next to the Status field to see the reason a campaign was cancelled.
   
5. On the Details page, search through the list of recipients in the table and check the status and email addresses of the users with delivery conflicts.
   
6. If the email status is Not Sent, check the Planned Send At column to verify that the campaign date and time correlate with the intended recipient’s time zone.
7. If the email status is Email Error, check the email address for misspellings. If the address looks accurate, try resending the email. A large volume of emails being received on the recipient server at the time the campaign emails were originally sent could have caused delivery issues.
   
8. If the issue persists even after resending the campaign emails, report the issue to the BullPhish Id support team for detailed investigation with reference to the campaign and email address with the issue.
9. If the email status on the Details page does not indicate Not Sent or Email Error, the email was sent to the recipient successfully but may have been missed or filtered by a third-party service on the recipient side.
10. After verifying that the email message has been sent to the end user, navigate as a recipient to the Outlook mailbox.
    

11. Navigate to Settings > Mail > Junk email > Safe senders and domains > Blocked senders and domains.
    
    

12. Verify that the sending profile on the campaign's Details page or the domain associated with the sending profile is not listed in Blocked senders and domains.
    
    
    
13. Verify that the sending profile or the domain associated with the sending profile is listed in Safe senders and domains.
    

5. NOTE  Custom configurations made in Outlook Mail Settings for the end user could create BullPhish ID issues resulting in messages not being delivered to the end user inbox.

Refer to the Microsoft Office 365: Automatically adding safe senders to Microsoft Outlook guide for details on how to create and run scripts to automatically add safe senders to each of your user's safe senders list in Microsoft Outlook. When you add new users to your Microsoft 365 environment, the BullPhish ID sending domains will be added to the new user's safe senders list automatically.

Identifying infrastructure configuration issues in Microsoft 365

Some custom rules may create additional delivery difficulties for the emails generated by BullPhish ID. To troubleshoot potential issues on the Microsoft Exchange server side, log into the administrative panel of your Microsoft Exchange account.

1. In the navigation pane, select Mail Flow and click Message Trace.
   
2. Search for the message in question and click it.
3. In the report window, the status should indicate the email message was Delivered.
   

The report will contain useful information about messages in Not Yet Delivered status, which could explain the root cause of the deliverability issue.

In addition, event-related Mail Flow rule executions which may cause email messages to be blocked or quarantined will be conveyed in the report.

NOTE  If you encounter a rule execution event preventing delivery of the email, check if you set up the safelisting of BullPhish ID domains and IPs for phishing simulation. Refer to the Microsoft Office 365 safelisting guide for more information.

Custom rules configured in Microsoft Exchange with high priority for all email messages could prevent emails from being delivered. If you have a custom setup, you will need to configure additional safelisting of BullPhish ID domains and IP addresses inside those rules as follows:

1. Identify the rule that is preventing delivery of the email message.
2. In the navigation pane, select Mail flow > Rules.
3. Click the applicable rule to open it.
4. In the form, configure the section Except if with values The sender domain is.
   
5. Add all the domains as instructed by the Microsoft Office 365 safelisting guide
   
6. Add the IP addresses per the Microsoft Office 365 safelisting guide .
   
   

Identifying email deliverability issues with Microsoft Defender

Once mail flow rules are correctly configured and safelisted according to the Microsoft Office 365 safelisting guide , you may still encounter deliverability issues. For example, maybe you can't find the email message in the Exchange Message trace flow. The root cause may be hidden somewhere in the configurations of Microsoft Defender or other security apps monitoring user inboxes.

Perform an in-depth analysis of the configuration of Microsoft Defender as follows:

1. Navigate to the administrative panel of Microsoft Defender.
2. In the navigation pane, select Email & collaboration > Explorer.
3. On the All email tab, search by sender or any other useful field.
   
4. In the Email results table, click the subject for the applicable email.
   
5. In the report panel, review the Delivery details.
   
   When safelisting is configured correctly and other services are configured such that they do not intercept email delivery, you should see that the email was delivered. In this example, the Primary Override: Source field indicates this email was allowed because of correct safelisting.
   

Overcoming blocked emails in Microsoft Defender

When deploying Microsoft Defender, several scenarios may come into play which can result in campaign emails not being delivered or blocked altogether. This section includes examples of these scenarios and how to overcome them.

Safelisting not applied-message blocked

In this example, we can see the email was blocked because it was identified as spam and the Primary Override: Source field shows None, meaning safelisting was not applied.

To investigate, check that the SMTP mail from address and Sender IP in the report matches the domain and IP safelisting information in the Microsoft Office 365 safelisting guide

Identifying custom domains

When domains associated with the sender differ from what you see on the Delivery details screen, it’s likely that you are using a custom domain which should also be properly safelisted according to details provided in the Microsoft Office 365 safelisting guide

Global block list

If the domain and IP address match values safelisted in your environment, you may still experience email delivery conflicts because of policy and rules configurations having higher priority under Block Lists. You can check this by navigating to Email & Collaboration > Policies & Rules >Threat policies >Tenant Allow/Block lists.

If the Tenant Allow/Block Lists table includes a Blocked action for the sender's email address or domain, it could be the root cause of the problem.

This is because the allow/block list has higher priority for Microsoft than BullPhish ID simulation rules. You can edit the list and redefine the action for the domain and email address from Block to Allow and it should solve the issue.

Block list for impersonation detection

Microsoft 365 Defender can be configured with a separate block list for Anti-phishing Impersonation Insight. Microsoft will block emails from sender domains that appear on its Impersonation Insight dashboard. To check domains for this issue, do the following:

1. Navigate to Email & Collaboration > Policies & Rules >Threat policies > Anti-phishing.
   
   
2. At the top of the page, click the View impersonations link.
   
3. On the Impersonation Insight page, check both the Domains and Users tabs. If the sender or domain associated with the affected email message is listed on either of these tabs, remove the address and or domain from the list. Any email address or domain appearing on this list will have higher priority for blocking than phishing simulation configurations and will need to be deleted.
   

IP address discrepancies with third-party security tools

The Microsoft 365 Defender Delivery Details report indicates if an IP address is different from the IP addresses listed in the Microsoft Office 365 safelisting guide . This will often be detected together with a failed SPF and DKIM check, along with any other security issues reported for the email message.

Often, the issue is related to the use of third-party phishing protection tools, such as Graphus, Mimecast, Barracuda, and others. Remedy this issue by performing the following:

1. Apply the safelisting recommendations directly inside the admin panel of the phishing tool in question.
2. Refer to Microsoft Office 365 safelisting guide.

After implementing the recommended safelisting, you may still see a different IP address in the email message report. Remedy this issue by performing the following: Configure a connector between Microsoft and the third-party cloud service by following recommendations on how to Manage mail flow using a third-party cloud service with Exchange Online.

In certain cases, you may not find email messages associated with your recipient or sender at all when reviewing the Microsoft 365 Defender Explorer page. This issue may arise if you use additional layers of security protection for end user inboxes in your third-party environment. These phishing protection tools could remove messages even without notifying Microsoft.

Requesting BullPhish ID support if still having issues with email deliverability

If you still have conflicts with email deliverability after reviewing this guide and following all the recommendations, request additional assistance from the BullPhish ID Support Team.

Before requesting additional assistance, gather the following information for a speedier resolution to your case:

• Reference your mailbox provider, Microsoft Outlook in this case.

• Indicate the type of your Microsoft environment, SaaS/On-premises, for example.

• If it is on-premises, mention the version and set of components: Microsoft Outlook, Microsoft Exchange, Microsoft Defender, for example.

• Indicate if additional third-party services, such Graphus, Mimecast, Barracuda, or others are used in the environment for protecting user inboxes.

• Gather screenshots of the configurations that you already applied using our guides.

• A copy of the report for email message from the Microsoft Exchange admin panel if available: https://admin.exchange.microsoft.com. Select Mail flow > Message trace. Search for and click the message in question. Click Copy report text.
  

• Download a copy of the email message from the thread explorer of Microsoft Defender if available: https://security.microsoft.com/threatexplorerv3. Select Email & collaboration > Explorer > All email. Search for and click the message in question.
  
  

• Provide additional information related to non-delivered email messages found in any third-party services .