Cireson Support Deploys Open Source SMLets Based Exchange Connector

Introduction SMLets Based Exchange Connector
Joe Burrows 

With the upcoming releases for Server 2019 and Service Manager (SCSM) 2019, the Cireson Support Portal was well over due for some love and updates. Given the Cireson Support Portal is used internally to dog food our upcoming releases, it’s important that our internal environment match what are customers are using. The goal was to move to a new SCSM environment on new server infrastructure in Azure and retire our 2012R2 VMs (RIP Metro Start Menu).

With the help of the Cireson SCSM Lifecycle Management app the proposed migration plan looked very simple: a side by side migration with a few button clicks, some Azure black magic, crossing of fingers and quick post migration testing before switching the load balancers to the new environment.

The first test: Notifications and Exchange Connector – mission critical feature and functionality for the support team.

Test Failed ☹: Action log notification was not sent, the exchange connector marked response as “null” on the comment.

Flashback to the old Exchange Connector bug that still hasn’t been fixed – a very popular alternative to scripting around this bug came to mind, something packed with many other features, gaining a lot of attention in the Cireson Community and the wider Service Manager community space… An SMLets based Exchange Connector.

The Solution and How it Helps
Adam Dzyacky 

The SMLets Exchange Connector is a completely free and open source PowerShell based alternative to the Microsoft Exchange Connector for Service Manager 2012 and up. It does everything the stock connector does plus fills a few process gaps, addresses some bugs, and since its inception almost two years ago continues to see frequent updates.

This new connector emerged out of equal parts necessity and the desire to bridge the gap between SCSM admins and SCSM developers. As such, its goals are simple: enable SCSM Administrators everywhere the opportunity to contribute to this conversation in a way they previously couldn’t and push the limits of what Service Manager can do through email.

But depending on where you are on the PowerShell spectrum of comfort, leveraging an open source solution not directly supported by any vendor may bring with it some challenges. So it’s worth addressing just what exactly makes up the SMLets Exchange Connector and how it was built. This new connector leverages:

• The Exchange Web Services *.dll (same one required for the stock connector)
• The Microsoft Active Directory module for PowerShell
• The open source SMLets module for working with Service Manager
• The open source MimeKit project for dealing with encrypted/digitally signed messages
• Cireson Analyst Portal Web APIs (only available for Cireson customers)

The take away here is that this new connector is built directly on top of fully supported Microsoft and Cireson technologies. SMLets (which is built using Microsoft’s *.dll files for SCSM) and Mimekit (recommended by Microsoft for working with encrypted messaging) while open source have had significant community support for years and were chosen specifically because of their open source nature.

This grants you the ability not just to review the PowerShell but to modify the connector to your organization’s content such as being able to address the aforementioned Null Comment “bug” and introduce new features such as:

• Filtering out attachments less than X file size (prevents signature graphics from being added to Work Items)
• Self-assign Work Items with the configurable [take] keyword
• Leave private comments by simply including the configurable #private on messages
• Post Announcements to SCSM, the Cireson SCSM portal, or both
• Vote on Review Activities on behalf of Active Directory Groups
• Leverage Azure Cognitive Services (Artificial Intelligence) to dynamically create Incidents or Service Requests based on Affected User sentiment and optionally set the respective Priority/Urgency/Impact combination
• Process email from multiple inboxes as a single inbox, so you only need one connector as opposed to deploying several for multiple inboxes (GitHub contribution provided by Tom Hendricks)
• Create Incidents, Service Requests, Changes, or Problems as the default Work Item
• And a whole lot more!

So when Cireson reached out to me, it was only to share that “We deployed it and we have it running in…”

Azure Automation Deployment Example
Joe Burrows

Why Azure Automation?

Having all our infrastructure in Azure. Our code and pipelines in AzureDevOps and already utilizing Log Analytics (OMS), it made logical sense to use Azure Automation for the execution of the SMLets exchange connector with Hybrid Runbook workers. This allowed for source control in a GIT repository, meaning any changes made are tracked and everything is kept nicely in a central familiar location (Who likes having scripts locally on different VMs anyway?).

Runbooks in Azure Automation can be triggered by a schedule or a webhook, a schedule has a minimum recurrence time of 1 hour. Every 1 hour is a little too long to wait between execution so we opted for the webhook trigger, another option if you wanted to use schedules rather than webhooks would be create multiple hourly schedules for each 5minute intervals. An ideal goal would be to author a custom SCSM connector that can trigger a webhook with a configurable interval rather than trigger from a Windows scheduled task.

Even if you’re not using Azure today, setting up Log Analytics and Azure Automation for your on-prem infrastructure is quick and easy. (See here to get started)

**Note: The Log Analytics client cannot be installed on an SCSM Management Server, so you will need to select a virtual machine that will be used to run the runbook and install the connector required pre-reqs.

High Level Steps

1. Download SMlets Exchange Connector script from GitHub
2. Configure hybrid runbook worker VM with pre-reqs (Smlets, AD Module, Exchange Web Services)
3. Configure smletsExchangeConnector.ps1 variables for the required settings
4. Test locally on hybrid worker VM
5. Configure hybrid worker agent
6. Setup and configure runbook in Azure Automation
7. Copy to Git repository and configure source control (Optional)

Step 1: Downloading the SMLets Exchange Connector 

1. Navigate to: https://github.com/AdhocAdam/smletsexchangeconnector/releases
2. Click the zip file as it corresponds to the version/release you wish to download.
3. Copy ZIP to the Server which will the Azure Automation Hybrid Runbook Worker.

Step 2: Configure Hybrid Runbook Worker VM with Pre-Reqs

1. Install SMLets on runbook worker VM. See the helpful guide here for installing SMLets on a Windows Server without the SCSM console
2. Install Active Directory module for Windows Powershell on runbook worker VM from server manager:
   
3. Install Exchange Web Services Managed API 1.2 on runbook worker VM (See link here for download)
   

Step 3: Configure smletsExchangeConnector.ps1 with required settings and configuration

**Note: These steps may change as future updates of smletExchangeConnector.ps1 are released**

1. Unblock the downloaded ZIP file, and extract
2. Open smletsExchangeConnector.ps1 with Powershell ISE – the script is well commented so it is easy to understand each variable in the script and what it does (Kudos to Adam and the contributors)
3. Configure SCSM variables – Go to line 109 and configure $scsmMgmtServer to point your SCSM management server, and any needed credentials on line 11:
   
4. Configure Exchange variables – Go to line 122 and configure variables required to connect to the Exchange mailbox. As we use Office365, settings used were like the below. (We will omit username and password after testing in a later step when publishing to AzureAutomation and replace with Get-AutomationPSCredential)
   
5. Configure SCSM and setting variables – Go to line 166+ and configure the default settings. These are self-explanatory and like what you configure in the out of box Microsoft Exchange connector
   
6. Additional features and configuration can be enabled from lines 190+ as described above in the solution

Step 4: Test locally on Hybrid worker VM

1. Send some test Email messages to the mailbox & execute the configured script
2. Confirm Email messages process and get moved out of inbox
3. Confirm SCSM tickets are created \ updated with the desired action log comment and attachments

Step 5:  Configure Hybrid Worker Agent

1. See here for instructions for registering the hybrid worker to an Azure Automation account.

Step 6: Setup and Configure runbook in Azure Automation

1. From the Azure Portal – open the Automation Account
2. From Credentials – create two credential assets, one for your SCSM workflow account, the other for your Exchange mailbox
   
3. From hybrid workers – select the hybrid worker agent group created from Step 5. Choose the Run As account to run as the SCSM-Workflow Credential Asset created above
   
4. From Runbooks – add a new runbook
5. Give the runbook a name and select PowerShell as the runbook type
   
6. Copy and paste the script tested in the earlier script in the canvas of the runbook
7. Update line 125 and 126 to remove username and password, as this stored on the Credential asset called ‘Support-O365Mailbox’ which the script will call at execution using Get-AutomationPSCredential
   
   
8. Save and publish runbook
9. Configure Webhook trigger – name the webhook and copy the URL:
   
   Configure webhook run settings to Use the hybrid worker group and click create
   
10. Create a scheduled task to invoke webhook and execute RunbookFrom the SCSM workflow server, create a new scheduled task to “Start a program”, every 5minutes (Or however often you want the runbook executed ) to Invoke-WebRequest to the webhook URL we created above.Program: powershell.exeArgs: -WindowStyle Hidden -command &{Invoke-WebRequest https://s1events.azure-automation.net/webhooks?token=RandomTokenFromUrlCopiedEarlier -Method post}
11. Test schedule task runs and executes runbook in AzureAutomation
    

Step 7: *Optional* Copy to Git Repository and Configure Source Control on Azure Automation Account

Azure automation gives the ability to sync with a GIT repository, either in GitHub, TFVS or AzureDevops (VSTS), this ensures all changes made to the code are tracked and stored in a central place.

If your new to Git checkout the below links:

What Is Git | Explore A Distributed Version Control Tool

Git Tutorial | Commands And Operations In Git

1. Copy the code from the runbook canvas & commit to a Git repository
2. From your Azure Automation Account, Select Source Control
3. Configure with the required settings
   

Conclusion
Adam Dzyacky 

Now that is how you integrate Joe! Whether it be Azure Automation, SMA, or even using Task Scheduler on your SCSM Workflow server – the SMLets Exchange Connector is a highly portable open source solution ready to enhance any Service Manager deployment.

So what about the future of the connector? What’s on the feature horizon? Is there a way to have an impact on what it can do? That would be a yes. After signing up for a GitHub account you can raise a Feature Request on the Repo by going to Issues -> New Issue -> Feature Request.

But what if your idea is a bit more direct? What if it’s cleaning up or introducing new PowerShell on the connector? While raising Feature Requests is one thing, coding in a public forum is another. So how exactly can you get started with making PowerShell level changes?

Are there some kind of instructions of “how to Github” and possibly specifically for the connector? That would be a resounding yes. The very same repository that hosts the connector also hosts the SMLets Exchange Connector Blog and features the article I want to contribute, but I have no idea how to use GitHub to help you come up to speed on GitHub knowledge and jump straight into the PowerShell.

When you are ready to join Cireson, other organizations, myself, and the contributors to the project you can find us on GitHub. But if you have questions, you can also reach out to us on the Cireson community.

More Resources

In addition this post, the following links can help you get started and learn more about Azure, runbook workers, GitHub, and the project itself:

http://blogs.catapultsystems.com/mdowst/archive/2013/10/29/install-smlets-on-windows-server-2012-without-scsm-console/

What Is Git | Explore A Distributed Version Control Tool

Git Tutorial | Commands And Operations In Git

This blog is co-authored by:

Adam Dzyacky
Senior Cloud Operations Engineer – BCU, SMLets Exchange Connector creator

Joe Burrows
DevOps Engineer – Cireson