Tuesday 5th September 2023, Microsoft will be conducting a 24-hour test of a new Root Certificate on their Microsoft Teams Direct Routing platform.
What does this mean for you?
Currently, all customer Session Border Controllers (SBCs) must store the Microsoft self-signed Baltimore Root Certificate in their trusted enterprise root certificate stores for TLS validation. Without it, SIP trunks to Microsoft Teams will fail to establish.
In 2025 this root certificate will expire. This left unmonitored has the potential to disconnect Teams calling for millions of users. Luckily Microsoft are on the ball and are addressing the problem before it arises.
Rather than renewing the existing root certificate and requiring all customers to update their SBCs with the new one everytime, Microsoft are transitioning to a globally trusted certificate authority, Digicert.
Transitioning to Digicert benefits all Direct Routing customers in the long term. As Digicert are a public, trusted certificate authority almost all Operating Systems include their root certificate chains within their distributions and update them automatically as part of their software update lifecycles. Whereas with Microsoft’s Baltimore certificate, only the Windows Operating System has this installed and updated automatically.
What Does The Test Entail?
Starting at 9:00am UTC on the 5th September, Microsoft will update their global Microsoft Teams Direct Routing endpoints to use a new certificate for SIP connections issued by Digicert. In doing so, this certificate will have a different trust chain to the one currently used by Direct Routing.
This is an “all-in” test with no option for customers to opt-out of the test, so if you use Direct Routing today, you will be impacted by this test.
What Do I Need To Do To Prepare?
You must make sure that your SBCs are updated to the latest version of supported firmware for Microsoft Teams Direct Routing. You can find the current certified firmware for your SBC here
The firmware should contain the latest certificate chains from all trusted certificate providers, including Digicert.
For added surety you can manually install the Digicert Global Root G2 by downloading it from the Digicert website.
Direct Link to PEM | Direct Link to CRT
Once downloaded, upload the root certificate to your SBC trusted root certificate store on all your Teams SBCs.
Can I Test This In Advance?
Yes, Microsoft have a test SIP endpoint you can validate your SIP trunks against. To do this, create a new SIP trunk using the current supported configuration for Teams Direct Routing.
Instead of connecting to sip.pstnhub.microsoft.com as the SIP endpoint, connect to sip.mspki.pstnhub.microsoft.com
You should enable SIP OPTIONS and monitor SIP messages for a 200 response from the Microsoft test endpoint to your SIP OPTIONS requests.
If you have configured your external firewall to only accept incoming SIP requests from specified Microsoft IPs, you will need to change this to support the entire Microsoft Teams Direct Routing IP ranges. These are 52.112.0.0/14 and 52.122.0.0/15 as requests during the test may originate from other SBC endpoints within Microsoft.
What Will Happen If I Don’t Do Anything?
Unless your SBC is from the prehistoric era, not much. As mentioned, most OS’s contain the root certificates of the most prevalent certificate authorities. As long as your system is up to date, your SBC should have the required trust in place.
But do you want to leave that to chance?
The worst case will be that your SBC will be producing SSL validation errors that will prevent decryption of the SIP TLS messages from Microsoft. This will prevent PSTN calls from establishing and you will suffer disruption.
Therefore, we highly recommend that you test in advance to mitigate any potential issues during the test.
What Should I Look Out For During The Test?
Hopefully the test will succeed and noone will be affected. However, with every test comes the risk of distruption. Here are some symptoms that you could experience if the test fails
- Intermittent or complete call setup failure – inbound and outbound calls fail to ring or answer successfully.
- Long duration call drops (30 minutes or more) – Calls appear to function normally but terminate unexpectedly after significant time.
- One-way audio – Only one person on the call can hear.
- Silence – both audio streams fail to establish and neither party can be heard.
- Call transfer / forwarding failures to complete
- Callers unable to join or dropped from call queues unexpectedly
What Should I Do If I Have A Problem?
The first thing you should do is check your configuration on your SBCs. Any TLS errors will be displayed on the real time monitor dashboard (Ribbon) or in the SIP Active Connections section (AudioCodes).
Alternatively you can capture and review your SBC SIP logs for TLS connection problems.
If you do not manage your own SBCs it is worth logging a case with your managed provider just incase they are experiencing an unrelated issue (coincidences do happen).
You should also check the Microsoft 365 Message Center for reports of any ongoing incidents and the public status page: https://status.office365.com/
No reports? Log a case with Microsoft or Report A Problem through the Teams client. The more cases raised to Microsoft by different customers will help them determine an issue quicker.
How To Opt-Out Of The Test?
This is a global test across all Microsoft Direct Routing endpoints. sip, sip2, and sip3.pstnhub.microsoft.com will all be using the new certificate during the test. There is no way for you to opt-out of the test.
Make sure that you are prepared.
Can I Mitigate The Potential Impact?
You may want to consider preparing routes on your SBC to re-route critical phone numbers to alternative solutions such as mobile or non-Teams integrated contact centers if you have them.
With Ribbon SBCs you can create these routes in advance and leave them deactivated. With AudioCodes, you can create them underneath the primary routes for numbers and move them up in the routing table if needed.
Alternatively you can setup automatic routing failover on your SBC so that this happens automatically.
Hopefully, all will go well and there will be no issues. Fingers crossed.
