Welcome to our final part of the series. Here we look at the statistics you need as a conference to measure success and what is available to you. We also share some of our lessons learnt that may help you if you are considering doing something similar.
Statistics
An event without any statistics isn’t going to last. You as organisers need conference statistics to use to understand where you succeeded and which areas need improvement. Not only that but you need these for your subsequent marketing materials for potential sponsors of your next event.
Speakers will also want to know how many people watched their session and what the feedback was. Sponsors who are speaking will want to know the names of the people who attended their sessions.
Using Microsoft Teams Activity Reports
Using the native reporting in the team and in the Teams Admin Center gives you a clear view of how many active users there have been in the conference team
These reports give you some high level insight into the participation within the team. The shortfall with these reports is that it is hard to distinguish between Attendee, Speaker, Exhibitor and Staff as well as their active times.
Being a 24 x 3 event, it would be nice to be able to see when people where active within the day and who.
AzureAD Logins
One metric we looked at is to find the last login time of the Attendee auth account (regforme). If we could get a report of who logged in on the days of the conference then at least we could drop down a level of detail where the Teams usage reports couldn’t provide the insight.
Of course this is open to interpretation as we do not have any way of measuring duration of participation, but at least it gave us some indication.
This script by Aaron Guilmette over at Undocumented Features helped us generate this report
$Filter = "UserType eq 'Guest'"
$Guests = Get-AzureADUser -All $true -Filter $Filter
Calculate users whose last STS refresh token value is 'n' past expiration
For example, if token expiration is 90 days, and StaleAgeInDays is 180, then
return objects that have a token age 270 days ago
$Today = (Get-Date)
$Global:StaleUsers = $Guests | ForEach-Object {
$TimeStamp = $_.RefreshTokensValidFromDateTime
$TimeStampString = $TimeStamp.ToString()
[int]$LogonAge = [math]::Round(($Today - $TimeStamp).TotalDays)
[int]$StaleAge = $MaxInactiveTime + $StaleAgeInDays
$User = $($_.Mail)
If ($LogonAge -ge $StaleAge)
{
[pscustomobject]@{
User = $($User)
ObjectID = $_.ObjectID
IsStale = "True"
LastLogon = $TimeStamp
DaysSinceLastLogon = $LogonAge
UserIsStaleAfterThisManyDays = $StaleAge
}
}
}
Session Attendances
One of the major advantages of requiring people to sign into our conference tenant was the ability to produce Teams Live Event attendance reports with real names in. This was valuable information especially to Exhibitors. It also serves as a barometer to us as organisers to view the most popular sessions.
Unfortunately the attendance report is a pretty ugly csv download. However a big shout out to Stuart Rideout for his Teams Live Event Power Bi Report
Unfortunately the attendance report is a pretty ugly csv download. However a big shout out to Stuart Rideout for his Teams Live Event Power Bi Dashboard
You can download his report here
This was a fantastic resource for us and saved us a lot of work having to work out and try and build something similar. This Dashboard allowed us to view the attendance over all events, plus drill down into each one and even see the Q&A asked during each event.
We modified it a little bit to add in our session feedback into the same Dashboard so that we have a single pane of glass insight into the event.
There where however gaps. Some of the sessions where Panel based and therefore, Teams meetings. We knew we wouldn’t be able to retrieve the session reports for those because the Attendance Report for Teams meetings can only be downloaded by the meeting organiser and then only during the meeting. As these where created by API, there was no hope of retrieving those.
Feedback
We collected feedback on a per session and overall event level. We did look at Microsoft Forms for this, but with the amount of sessions it was hard to create something in Forms that allowed people to select from a pre-selected list of sessions, or even just intelligently know which session the feedback was for. We where never going to create 230 individual Microsoft Forms…
To do this we used our WordPress website and a plugin called Formidable Forms. This allowed us to use cascading select fields we could let people choose from to get to their session. At the time it seemed the best way to do it, but as we have learnt since we have found an even easier way to do this. Watch out for our blog on that!
We collected the session feedback and inserted into the Power Bi report
Lessons Learnt
For a first attempt at actually organising and running a conference of any description along with having to adapt to the new situation we found ourselves in, we think we did a good job.
Yes, we made mistakes and missed a few things, but a lot went right. A lot of that was down to the way the Community surrounded us and helped out in ways we probably won’t really fully know.
Teams performed amazingly well, not one meeting issue or technical glitch and what better way of showcasing Microsoft Teams than to use it to deliver a Technical Conference using Teams.
However, this was a learning experience for us all. Off the back of this event now we have a clearer picture of what works and where improvements need to be made.
What Works
- Teams Live Events is perfect for Online Conferences and mixed with interactive breakout meetings gives Attendee’s and Speakers to connect with each other and that contributed a huge amount to the “together” sentiment felt by many
- Virtual Reality was an amazing experience and with the help of Jon Mckinney we where the first technical conference to bring this to an event and dared to be different. It was such a success that other conferences have since decided that this should be part of their event. Who would have thought that part-time conference organisers could make full-time conferences look twice… 🙂
- People Management was a huge part of the success of this event. Without going through everything 4 or 5 times and making sure that we pestered everyone for a response to ensure that they knew what they needed to do, I don’t think it would have run so smoothly.
- The Power Automate posting adaptive cards was well received with people commenting that once they found their way into the team, joining sessions was really easy and the cards helped them drop in and out of the conference between work and easily see what is going on at that time.
What Didn’t
- Guest Access was a hurdle many could not get over. A lot of the sentiment towards this is down to people’s patience, but the user experience didn’t help the situation. We estimated that we probably lost about 10% of our audience as a result.
- Booth Channels for Exhibitors we thought where a great idea. However, we didn’t see the interaction we had hoped for in them. Perhaps this was down to the pace of the content, or that Attendees will not naturally reach out themselves in the public manner of a channel. Exhibitors had more traction by attending sessions and breakouts.
- Attendees and Speakers self moderated the channels, but Exhibitors became a little spammy in posting adverts in all kinds of channels. This wasn’t their fault, they where just trying to find the best forum to announce their participation. However, it did act as a turn off for what the channel where supposed to be about. If we did this again, we would have less channels and more time between sessions for people to explore the team.
Doing it Again?
We are really an in-person conference, that is where we want to be. However, we have to be realistic in that online conference format is here to stay. At the very least as a hybrid event. Stay tuned for our announcements closer to March…
Doing this again though we would approach things a little differently and we would work to remove the joining friction people experienced and keep the best parts of what did work and rework perhaps the format into more of a Community & Exhibitor inclusive format that gives everyone the best opportunities to collaborate.
Thank you to Everyone Who Took Part
I will part away from this blog series by leaving some of the amazing comments people left on Social Media…
