Should you still disable legacy sign-in, even if using Conditional Access?

It's no secret that Conditional Access is the pillar of any good security plan and one of the best features available as part of the Microsoft 365 Business Premium license set aimed at small businesses, but it's not the be all and end all and other security measures should be implemented around it.

I recently attended a session with the Microsoft 365 Security and Compliance User Group and asked the question "Should I still be disabling legacy authentication at the tenant level, even if I have it blocked via Conditional Access, I want to stop seeing failed sign-ins, because CA applies after authentication I continue to get the failed sign-in attempts and it's annoying!" to which the response was "go test it and let us know" - So here's my research and findings around my original question!

First things first, lets get some legacy auth sign in logs!

I first had to enable legacy authentication for my test user account in order to test, I did this from the Microsoft 365 Admin portal but you can also do this with PowerShell as well.

In order to get some logs I kept it simple and just used PowerShell to connect to my test user account and sent a test email over SMTP using the below PowerShell commands, just remember to update your From and To addresses.

$msolcred = get-credential

Send-MailMessage –From user@domain.com –To user@hotmail.com –Subject “Test Email” –Body “Test SMTP Relay Service” -SmtpServer smtp.office365.com -Credential $msolcred -UseSsl -Port 587

Great, now I have some sign-in logs using legacy auth and I can continue to use that sign-in protocol.

Now we need to create the Conditional Access policies to block legacy authentication, after which I'm expecting to see failed sign-in attempts in the Azure AD sign-in logs. Normally I would create custom CA policies but we'll use the new 'create policy from template' preview option for speed and ease, keep it in report only mode until you are happy to enable the policy.

I can see after reviewing the policy everything is configured as we would want it, so I go ahead and enable it for all users.

After 5-10 minutes I attempt to authenticate again, this time using SMTP Test Tool (gmass.co) and just as expected we see the failed sign-in within the Azure AD sign-in logs section.

And herein lies my original question, if we disable legacy auth at the tenant or user level, do we still see these failed sign in attempts in the Azure sign-in logs, so now time to disable it for my test user in the Microsoft 365 Admin portal.

I wait another 5-10 minutes for the changes to take effect and then test again, this time with both PowerShell and SMTP Test Tool (gmass.co).

Bingo! What would have been two failed sign in attempts, now lets go back and check the Azure sign-in logs.

Sure enough the only sign-ins we see were those that were made when SMTP authentication was enabled for the user and with Conditional Access blocking one of the sign-ins.

This is good information to know because we now know that by disabling legacy authentication protocols at the user or ideally, at the tenant level we can:

• Reduce noise in the Azure AD sign-in logs

• Stop user accounts getting locked out due to brute force attacks

• Stop the potential for successful password spray attacks

After playing out the scenarios the above makes total sense, because Conditional Access policies don't apply until after authentication, we see these in the Azure AD sign-in logs, but when we disable the legacy authentication protocol altogether sign-in attempts don't even make it that far, altogether providing a more secure and cleaner environment.

So, the answer is yes! We should be disabling legacy authentication protocols in Microsoft 365, even if you are already blocking them using Conditional Access policies.