<style>p { margin: 0; }span.fr-emoticon.fr-emoticon-img { background-repeat: no-repeat !important; font-size: inherit; height: 1em; width: 1em; min-height: 20px; min-width: 20px; display: inline-block; margin: -0.1em 0.1em 0.1em; line-height: 1; vertical-align: middle; } span.fr-emoticon { font-weight: normal; font-family: "Apple Color Emoji", "Segoe UI Emoji", "NotoColorEmoji", "Segoe UI Symbol", "Android Emoji", "EmojiSymbols"; display: inline; line-height: 0; } blockquote { border-left: solid 2px #5e35b1; color: #5e35b1; margin-left:0; padding-left:5px;}blockquote blockquote{ border-color: #00bcd4; color: #00bcd4;}blockquote blockquote blockquote{ border-color: #43a047; color: #43a047;} table.grid{ border-collapse: collapse;} table.grid td, table.grid th { border: 1px solid #ddd;} .fr-fic.fr-dib{ display: block; margin: 5px auto;}.fr-fic.fr-dib.fr-fir{ text-align: right; margin: 5px 0 5px auto;}.fr-fic.fr-dib.fr-fil{ text-align: left; margin: 5px auto 5px 0;}.fr-fic.fr-dii{ float: none; margin: 5px auto;}.fr-fic.fr-dii.fr-fil{ float: left; margin: 5px auto;}.fr-fic.fr-dii.fr-fir{ float: right; margin: 5px auto;}img.fr-dib.fr-fir { margin-right: 0; text-align: right;}img.fr-dib.fr-fil { margin-left: 0; text-align: left;}img.fr-dib { margin: 5px auto; display: block; float: none;}img.fr-bordered { box-sizing: content-box; border: solid 5px #CCC;}img.fr-shadow { box-shadow: 10px 10px 5px 0px #cccccc;}img.fr-rounded { border-radius: 10px; -moz-border-radius: 10px; -webkit-border-radius: 10px; -moz-background-clip: padding; -webkit-background-clip: padding-box; background-clip: padding-box;}</style><style>
p {
margin: 0;
}
span.fr-emoticon.fr-emoticon-img {
background-repeat: no-repeat !important; font-size: inherit; height: 1em; width: 1em; min-height: 20px; min-width: 20px; display: inline-block; margin: -0.1em 0.1em 0.1em; line-height: 1; vertical-align: middle;
}
span.fr-emoticon {
font-weight: normal; font-family: "Apple Color Emoji", "Segoe UI Emoji", "NotoColorEmoji", "Segoe UI Symbol", "Android Emoji", "EmojiSymbols"; display: inline; line-height: 0;
}
blockquote {
border-left: solid 2px #5e35b1; color: #5e35b1; margin-left: 0; padding-left: 5px;
}
blockquote blockquote {
border-color: #00bcd4; color: #00bcd4;
}
blockquote blockquote blockquote {
border-color: #43a047; color: #43a047;
}
table.grid {
border-collapse: collapse;
}
table.grid td,
table.grid th {
border: 1px solid #ddd;
}
.fr-fic.fr-dib {
display: block; margin: 5px auto;
}
.fr-fic.fr-dib.fr-fir {
text-align: right; margin: 5px 0 5px auto;
}
.fr-fic.fr-dib.fr-fil {
text-align: left; margin: 5px auto 5px 0;
}
.fr-fic.fr-dii {
float: none; margin: 5px auto;
}
.fr-fic.fr-dii.fr-fil {
float: left; margin: 5px auto;
}
.fr-fic.fr-dii.fr-fir {
float: right; margin: 5px auto;
}
img.fr-dib.fr-fir {
margin-right: 0; text-align: right;
}
img.fr-dib.fr-fil {
margin-left: 0; text-align: left;
}
img.fr-dib {
margin: 5px auto; display: block; float: none;
}
img.fr-bordered {
box-sizing: content-box; border: solid 5px #CCC;
}
img.fr-shadow {
box-shadow: 10px 10px 5px 0px #cccccc;
}
img.fr-rounded {
border-radius: 10px; -moz-border-radius: 10px; -webkit-border-radius: 10px; -moz-background-clip: padding; -webkit-background-clip: padding-box; background-clip: padding-box;
}
</style><p><strong>In this guide we will cover:</strong></p><p><strong>- What is Backend Services Monitoring? </strong></p><p><strong>- Navigating the Backend Services Monitoring area</strong></p><p data-pasted="true"><strong>- Backend Service Monitoring for UAT Instances</strong></p><p><strong>- Common Errors</strong></p><p><br></p><p><br></p><p><strong>Related Guides:</strong></p><ul><li style="font-weight: bold;"><a href="https://usehalo.com/haloitsm/guides/2731/" target="_blank" rel="noopener noreferrer"><strong>Log Retention</strong></a></li></ul><p><br></p><p><br></p><p><strong><span style="font-size: 14pt;">What is Backend Services Monitoring? </span></strong></p><p>Backend services monitoring allows you to monitor services, automations and events that run on the backend of Halo. <span style="color: rgb(0, 0, 0);">This includes integration syncs, incoming and outgoing email, notifications, workflow automations, webhook requests and responses. </span>The backend services monitoring area allows you to see logs relating to these processes These logs are helpful in identifying and resolving issues. In this guide we will run through what each section of the backend services area shows and how to interpret the logs here.</p><p><br></p><p><br></p><p><strong><span style="font-size: 14pt;">Navigating the Backend Services Area</span></strong></p><p>To get to this area head to configuration > advanced settings, here you will see the section 'backend services', select the button 'Backend Service Monitoring'. </p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6Ijk5YTc1NmE4LWU2NGMtNDg5OS05NzJkLTMwMDhkNThiNDQ3YyJ9.ocoRc5AwYTVfCCISTJ4C2z4WzWjbLlZgrN2r-mMeToM" class="fr-fic fr-fil fr-dib" width="903" height="497"></p><p><strong><span style="font-size: 10pt;">Fig 1. Backend Services Monitoring button</span></strong></p><p><br></p><p>When selected you will be taken to the backend services monitoring area. Here, data is stored under various tabs, we will go through what is stored in each of these tabs.</p><p><br></p><p><strong><span style="font-size: 10pt;">Fig 2. Backend Service Monitoring area</span></strong></p><p><br></p><p><strong><span style="font-size: 12pt;">Configuration Tab</span></strong></p><p>At the top of the page you will notice a message explaining what the backend services monitoring area shows, there is also a link to show you which tasks will be processed the next time the scheduler runs. Will we look at this further in the section 'Task scheduler log'. </p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6Ijk2ZGZiOWE4LWI3Y2QtNGU4Yi1hZTgxLTNkMTIzODgxNTIzZCJ9.zk2e6O3WbICYpyZxTXAimDse63ngQx7MA5VZGB1jsXk" class="fr-fic fr-fil fr-dib" width="1390" style="width: 1392px; height: 784.728px;" height="785"></p><p><strong><span style="font-size: 10pt;">Fig 3. Configuration tab in Backend services monitoring </span></strong></p><p><br></p><p><strong>Log level</strong> - The dropdown here determines how much information is shown in each of the logs. Options are ordered by the most to the least amount of information that will show in the log.</p><ul><li>Trace - This option will cause the most information to be shown in the log.</li><li>Critical - This option will cause the least information to be shown in the logs.</li></ul><p style="margin-left: 20px;">By default this is set to 'Information', we recommend keeping the log level set to this. Increasing the information shown will increase load on the server and can lead to performance issues. If you are experiencing issues and require more log information the log level can be increased temporarily (to Debug) and changed back once a few events have been processed with additional log information. However, the 'Information' log level should provide enough information for troubleshooting. </p><p><br></p><p><strong>Pause all background services</strong> - When enabled background tasks will stop running, such as outgoing email, automations, scheduled reports, notifications etc. but each type of service can still be manually processed from each tab. </p><ul><li>This is used when you need to stop an automation/event being processed. For example, if you have queued many emails to be sent (incorrectly) or have notifications being triggered incorrectly and need to pause the sending/processing of these while you resolve the root cause. </li></ul><p style="margin-left: 20px;">When services are resumed after being paused, any paused tasks will be automatically processed. Any tasks/automations that you would not like to be processed will need to be removed while the services are paused. </p><p><br></p><p><strong>Enable daily clean up of ephemeral records</strong> - When enabled ephemeral records (temporary logs) will be cleared out daily. This is required to prevent bloating in your database, do not disable this setting. This is only ever disabled when logs need to be retained for longer periods temporarily, such as to troubleshoot issues. </p><p><br></p><p><strong>Minimum severity level for error notifications</strong> - This setting determines which error level notifications you are notified about. The lower the value the higher the severity level the issue needs to be for you to be notified. For more information on this see our guide <a data-fr-linked="true" href="https://usehalo.com/haloitsm/guides/1504/" id="isPasted" target="_blank" rel="noopener noreferrer">here</a>. </p><p><br></p><p><strong>Snooze Monitoring Events</strong> - This setting allows you to 'snooze' monitoring events based on severity. When events are snoozed agents will not be notified of them until the selected date. </p><p><br></p><p><strong>Maintenance</strong></p><p>Here are various buttons that will clear unsent and stuck actions/emails/notifications. </p><p><br></p><p>By clearing these entities anything that has not been processed will be deleted and you will not be able to process them at a later date. These buttons are used when you have something unprocessed 'clogging up' the services, such as if lots of outgoing emails have failed to process due to an incorrectly setup mailbox.</p><p><br></p><p>If you have backend services notifications enabled the notification will continue to trigger if you still have a failed entity in your logs (depending on your severity level for error level notifications). Therefore, you can clear stuck actions/emails from the store so they are cleared from the log and to stop the notification triggering. </p><p><br></p><p><strong><span style="font-size: 12pt;">Halo Integrator Log</span></strong></p><p>Under this tab you will see a log for each time the Halo Integrator has run, i.e. each time the Halo integrator has pulled information from another application into Halo. A log will contain information about what integrations are being processed, what data is attempting to be imported, the outcome of the sync and any error messages. </p><p><br></p><p><strong><em>Note: The Halo integrator executes scheduled integration imports, for more information on this see our article <a data-fr-linked="true" href="https://usehalo.com/haloitsm/guides/1062/" id="isPasted" target="_blank" rel="noopener noreferrer">Halo Integrator</a>. </em></strong></p><p><br></p><p>The Halo integrator is most commonly used in conjunction with application integrations. When troubleshooting issues with integrations this is a great source of knowledge as you can see what data is being pulled in on each run, when the integrator has failed and why. </p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjJiNDBkNDBiLTZjZjItNDc1Yi1hZmMzLTMyNWI5M2FkZjY2MiJ9.5bDbeffUGgvZsA8g6lmSbiEwT4AAkJ9xhB-YMycQV-U" class="fr-fic fr-fil fr-dib" width="1219" style="width: 1221px; height: 469.387px;" height="469"></p><p><strong><span style="font-size: 10pt;">Fig 4. Halo integrator Logs</span></strong></p><p><br></p><p><strong>Interpreting Halo Integrator Logs</strong></p><p>Click into the log to see details of the integrator run. </p><p><br></p><p>The information that you see in the log will depend on the integration that is being processed however typically a log will show:</p><ul><li>The time the sync has started for a particular integration </li><li>What is being processed as part of the sync (customers/devices/users/licences)</li><li>Which parts of the sync have been successful</li><li>Which parts of the sync have failed, along with an error message</li></ul><p>This is useful as it can detail which specific entities have failed to sync, for example when a RMM integration has synced the log will detail exactly which assets have updated and which have failed. Allowing you to identify the 'problem' assets. </p><p><br></p><p>Lets take a look at some example logs.</p><p><br></p><p>Log for a successful sync</p><p><br></p><p><strong><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6ImU0N2QzNDc0LWZhNTctNDU1Yy1hMjA3LWEwOTJhYWNhYTQ2OCJ9.54odDNI4o9YwNG912F0jzgHVC8R6YrYsZi27GRgue1Y" class="fr-fic fr-fil fr-dib" width="860" height="480"></strong></p><p><strong><span style="font-size: 10pt;">Fig 5. Halo integrator log for Azure AD with no errors</span></strong></p><p><br></p><p>In figure 5 you can see an example log for the Halo integrator, personal information has been redacted from this log. Let's breakdown what the log shows:</p><p style="margin-left: 20px;">Red = The integration that the Halo integrator is running for (and the time this started) </p><p style="margin-left: 20px;">Yellow = The tenant within the Integration that the Integrator is processing.</p><p style="margin-left: 20px;">Purple = The entity the integrator is syncing. In this case it is agents but the entity will vary based on the integration. </p><p style="margin-left: 20px;">Blue = The number of mappings for the entity that has been found and is syncing. Mappings will be configured for certain entities. </p><p style="margin-left: 20px;">Orange = Outcome of the sync for a specific agent. The agent email address will show here to show which agents have synced successfully, in figure 4 this information has been redacted. </p><p style="margin-left: 20px;">Green = The integration the sync has ended for and the time it has ended.</p><p><br></p><p>Here in the log I can see a sync for the integration Azure active directory has started at 09:38:06, one tenant was found in this integration. The integrator started to sync the entity 'agents' (in this case only this entity has been set to sync for this integration but often multiple entities will sync). One agent mapping was found, if more mappings were configured the count would reflect here. Five agents have been found in Azure AD and all five have been updated/created successfully. The processing for agents finished, completing the sync for this tenant. The integration then checked for any users in Azure AD that need deactivating, as users have not been set to sync for this integration no users have been found and none have been made inactive. The integration sync finished at 09:38:12. </p><p><br></p><p>Preview of logs will have a timestamp but will not state which integration the log is for as various integration data can be imported for multiple integrations in one run.</p><p><br></p><p><strong>How to find the Log you need for Troubleshooting</strong></p><p>Although the log preview screen will not show the integration the log is for you can find the log for a specific integration based on the timestamp of the log. </p><p><br></p><p>For example, if I am having issues syncing data for the NinjaOne integration I will want to see the log containing the sync for this integration. Head to configuration > integrations > Halo integrator > hosted integrator schedule. Here you can see the schedule (times) the Halo integrator is set to run for each integration. </p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjEzNjJjNDNmLTRlODEtNDU4My05ODg5LTdmNTgwODRkMWFiNiJ9.nLYQeJ8xHzRxpnJZaqKkfJQn1vD6gtqjtW6rU_5aXR8" class="fr-fic fr-fil fr-dib" width="1225" style="width: 1227px; height: 320.588px;" height="321"></p><p><strong><span style="font-size: 10pt;">Fig 6. Hosted integrator schedule </span></strong></p><p><br></p><p>We can see in figure 6 the integrator last run for the integration NinjaOne was on 03/03/25 at 09:37. Therefore I will need to find the log with this timestamp in the log list, shown in figure 7. </p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6Ijk3MDYyZjAxLWEwZWEtNGFkZi05MjI5LWM1NTAzMmYxNzAxMCJ9.Jlr_Bd0VzXAvbDDOQBK4U86L9l_uHMujVHqdZAgo0z4" class="fr-fic fr-fil fr-dib" width="1216" style="width: 1218px; height: 562.759px;" height="563"></p><p><strong><span style="font-size: 10pt;">Fig 7. Associated log in Halo integrator list</span></strong></p><p><br></p><p><strong><em>Note: The integrator has run for several integrations at this time therefore the log will contain information for all integrations scheduled to run. The sync for each integration will occur sequentially and the start of the sync for each integration will be denoted in the log. </em></strong></p><p><br></p><p><strong><span style="font-size: 12pt;">Task scheduler Log</span></strong></p><p>Under this tab you will see a log for all scheduled tasks that have been queued to run. Scheduled tasks include:</p><ul><li>Creation of scheduled tickets</li><li>Creation of automated tickets </li><li>Creation of recurring appointments</li><li>Indexing (tickets and articles) </li><li>Automated actions and emails</li><li>Checking live chats for abandoned status</li></ul><p><br></p><p>The task scheduler queues outstanding automations, events, scheduled occurrences, outgoing mail and processes daily and miscellaneous tasks. The task scheduler will run, checking the status of certain entities in the database and processing outstanding tasks every 5 minutes. For example, the task scheduler will check the database every 5 minutes for the status of agreements (customer and supplier), if an automated ticket is set to create based on agreement criteria (e.g. contract expires in 7 days) the automated ticket will be created once the task scheduler has identified there is an agreement that meets this criteria. </p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjNjNGJhOTVhLTUyOTctNGFjZS05M2IxLWRlYzcyNzZmODhmOSJ9.F3k_gPW9ENuV0kYhQu4luMZZ66RzKqxWThge86_ZE6c" class="fr-fic fr-fil fr-dib" width="1216" height="657"></p><p><strong><span style="font-size: 10pt;">Fig 8. Task scheduler log</span></strong></p><p><br></p><p>The log preview screen will show the timestamp of the scheduled tasks and if they were forced or not. An entry will be logged as 'Forced' if tasks were forced to run using the 'Run Task Scheduler Now' button (top of the screen). </p><p><br></p><p>'Run Task Scheduler Now' button - When this button is used any scheduled tasks will be forced to run right away rather than having to wait for the scheduled time. This is used when you need a task to be carried out right away, for example, if testing automated/scheduled tickets or testing the pre-pay auto top up feature and you need to automatically trigger the top up. </p><p><br></p><p>To see which tasks will be processed next time the task scheduler runs head to the 'Configuration' tab and follow the link following 'Tasks that will be processed the next time the task scheduler runs can be viewed at'.</p><p><br></p><p>To view a log click into a log entry and head to the 'Log' tab. Here you will see what tasks/scans have been run. The outcome of the scan will not appear in the log here as the task scheduler does not execute the automations, just queue's them. The outcome will appear in either the outgoing log, event log or automation log.</p><p><br></p><p><strong><span style="font-size: 12pt;">Incoming Log</span></strong></p><p>Here you can see a log of all the emails that have come into your connected mailboxes, the log preview will show who the email is from and the subject. Logs will be sorted chronologically, regardless of the mailbox they were sent into, for mailbox specific logs see the 'Inbound Log' tab, which is found within the configuration of each mailbox.</p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjM5NDIxMDUzLThkNTItNGUxNC05MDdhLWZkMGRlMmY0YzgzYSJ9.kH7ksGaJbIRLg1iPZ44Yu8I-PWXlg3ltqaTpEE45Csk" class="fr-fic fr-fil fr-dib" width="1217" style="width: 1219px; height: 599.776px;" height="600"></p><p><strong><span style="font-size: 10pt;">Fig 9. Incoming log </span></strong></p><p><br></p><p>Click into a log entry to see more detail. When the log is opened head to the 'requests' tab, there will be a request entry for each time the mail has attempted to be processed. A mail will be attempted to process three times, if it fails all three it will not be processed unless an agent prompts a retry. You can click into any request entry to see more detail about the request/why it failed. </p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjE5ZTQzNTA3LTU3MWUtNGI2Zi04YTE3LTBkZDVhNDc5Nzc2YyJ9.P3ObpZjE5UBkHmi7bU7H2Wbt4ty4ftxZCMFxo7ude6w" class="fr-fic fr-fil fr-dib" width="1222" style="width: 1224px; height: 303.688px;" height="304"></p><p><strong><span style="font-size: 10pt;">Fig 10. Requests for mail that has failed to process</span></strong></p><p><br></p><p>Once a request is selected head to the 'Response' tab to see the outcome of the mail/request. If the mail has failed to be processed the error message will display here. </p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjkzMDgwNWRjLTI5ODUtNDIxMC1hZjU4LWRjMGFjMDI0MmVlOSJ9.BkwerAx4hcDCJTaIM1TjY8ab2BUgplD9m92nGExNr6s" class="fr-fic fr-fil fr-dib" width="1215" style="width: 1217px; height: 374.539px;" height="375"></p><p><strong><span style="font-size: 10pt;">Fig 11. Response of failed mail</span></strong></p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjZiYmViNWNmLTFhMmYtNGZmMS1iYTRmLWFjZjBjNDRlZjAwZiJ9._xuQywyLzIaLojpGTL68DEKYL5TYV6itNetladABbUs" class="fr-fic fr-fil fr-dib" width="1050" style="width: 1052px; height: 407.411px;" height="407"></p><p><strong><span style="font-size: 10pt;">Fig 12. Response of successful mail </span></strong></p><p><br></p><p>If the mail has been processed successfully a 'Log' tab will also show, here you can find a breakdown of how the system has processed the mail. </p><p><br></p><p>These logs can be used to troubleshoot issues with incoming emails not processing, primarily using the error messages against the request. </p><p><br></p><p>Once you have identified the cause of the issue and resolved this you can attempt processing of the emails again using the 'Retry' button against the log. This can also be done in bulk by selecting multiple logs from the log preview screen, the 'Retry' action will be available when hovering over the 'Edit' button. </p><p><br></p><p><strong><span style="font-size: 12pt;">Outgoing Log </span></strong></p><p id="isPasted">Here you can see a log of all the emails that are being sent out of Halo using your connected mailboxes, this includes both emails that are sent by agents manually and any automated emails. The log preview will show when the message was sent and the subject. Logs will be sorted chronologically, regardless of the mailbox they were sent into, however you can filter the logs using the mailbox ID, for mailbox specific logs see the 'Outbound Log' tab against the mailbox. </p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6ImJhZmM1ZWQ4LWE0ZGMtNDNhZS05NDM2LTRjZGI2ZThhNjI3OSJ9.LTtwJzGHCqtSuj7-4Xzs61FrH7tRLbxDltP9koXlsQI" class="fr-fic fr-fil fr-dib" width="1215" style="width: 1217px; height: 453.27px;" height="453"></p><p><strong><span style="font-size: 10pt;">Fig 13. Outgoing Log</span></strong></p><p><br></p><p id="isPasted">Click into a log entry to see more detail. When the log is opened you will see the details of the email that has come into the mailbox, who it has been sent to, the subject and the body of the email. Under the 'Log' tab you will see the processing details for this mail, if the mail has failed an error message indicating the reason for failure will show here. The 'Attempts' tab will show the number of times the mail has been attempted to be processed, processing will only be attempted 3 times but can be prompted again by the agent using the 'Resend' button. </p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjMzNzk2YWYzLTY0N2UtNGFhYS04M2Q3LTk2NDI5NjJhMmM3ZSJ9.MxM8n3PNCcs3ItPwGTTt4G2gdDYnwtrrsOoyvxyp03U" class="fr-fic fr-fil fr-dib" width="1219" style="width: 1221px; height: 287.988px;" height="288"></p><p><br></p><p><strong><span style="font-size: 10pt;">Fig 14. Log of failed outgoing email</span></strong></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjVkYTZmOTA1LTE1Y2YtNGFlYy1iMjlkLWJjMWI0NmUzNmVlNCJ9.PWSjVBxwV-5ZJUgsNfouukWXomsovX4_qblOhBgbRmk" class="fr-fic fr-fil fr-dib" width="878" style="width: 880px; height: 327.211px;" height="327"></p><p><strong><span style="font-size: 10pt;">Fig 15. Log of successful outgoing email</span></strong></p><p><br></p><p>These logs can be used to troubleshoot issues with outgoing emails not processing, primarily using the error messages against the request. </p><p><br></p><p>Once you have identified the cause of the issue and resolved this you can attempt processing of the emails again using the 'Resend' button against the log. This can also be done in bulk by selecting multiple logs from the log preview screen, the 'Resend' action will be available when hovering over the 'Edit' button. </p><p><br></p><p><strong><span style="font-size: 12pt;">Event Log </span></strong></p><p>Here you can see logs for each notification and webhook that have been triggered by an event. If the notification is sent to send an email the log for the email being sent out can be viewed under the 'outgoing Log' tab.</p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6Ijg0MDU0Yjk3LTRmODAtNGNhNS1iZjZlLTk4MDQxZTg0NDMwYiJ9.kU56jyh0bP9dD7Wm6f_1wA9pLtv5ZmOI7By2iPMK6E0" class="fr-fic fr-fil fr-dib" width="1220" style="width: 1222px; height: 407.058px;" height="407"></p><p><strong><span style="font-size: 10pt;">Fig 16. Event Log Tab</span></strong></p><p><br></p><p>When previewing the logs you will see description of the event that occurred as well as the number of notifications/webhooks that were created by the event (creation count). Click a log entry to see the log in more detail and head to the Log tab. </p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjkyOWRjMDJmLTFlOWUtNGRmZS04OWI4LTU1NzkwZWEwZmNkOSJ9.Mhuvtyrpw00aJaEbaImHMudXYnyTzbnzk2MVtCwP8Fs" class="fr-fic fr-fil fr-dib" width="1045" style="width: 1047px; height: 416.341px;" height="416"></p><p><strong><span style="font-size: 10pt;">Fig 17. Event log example - notification </span></strong></p><p><br></p><p>The figure 17 example shows the log for the event 'document uploaded to ticket' triggering the creation of a notification. </p><p><br></p><p>"Starting conditions check for unid=12" - This tells us the conditions for notification with ID = 12 were met. This notification was sent. </p><p>"Notification (TechOld) will be added for agent (3)" - This tells us the notification is a pop-up notification (TechOld=pop-up in agent app) was sent to agent with ID= 3. </p><p>"EscMsg records to create=1" - This tells us one notification record was created. </p><p><br></p><p id="isPasted">To find the ID of a notification head to configuration > notifications> notifications> select a notification, the URL here will contain the notification ID. </p><p><br></p><p>This can be used to see if a notification was triggered for a particular agent at a certain time.</p><p><br></p><p>Let's take a look at a log example when an event to trigger a webhook occurs. </p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjZjYTdhNDM1LTNhNDItNGI5Yy1hNDUyLTdhMWIyMTc1N2JjNCJ9.yuWV3AgWCQT_rKDw_LjI4Rgn2UH8e0vyP_0aXJUOvQo" class="fr-fic fr-fil fr-dib" width="1093" style="width: 1095px; height: 454.846px;" height="455"></p><p><strong><span style="font-size: 10pt;">Fig 18. Event Log example - webhook</span></strong></p><p><br></p><p>The figure 18 example show the log for the event 'Invoice updated' triggering the post of a webhook. In this example a single event has triggered two webhooks. </p><p><br></p><p>Notification (WebHook) will be added for webhook (xxx94) - This line of the log tells us criteria to fire webhook with ID xxx94 have been met, so this webhook has been created and posted. </p><p>Notification (WebHook) will be added for webhook (xxx4c) - This line of the log tells us criteria to fire webhook with ID xxx4c have been met, so this webhook has been created and posted. </p><p>EscMsg records to create=2 - This line of the log tells us two webhooks have been posted. </p><p><br></p><p>To find the ID of a webhook head to configuration > integrations > webhooks > select a webhook, the URL here will contain the webhook ID. </p><p><br></p><p><strong><span style="font-size: 12pt;">Automation Log </span></strong></p><p>Here, you can see logs for all automations that take place in your instance. This includes workflow automations, runbooks, scheduled system actions, AI insights, SLA auto-release. Ticket acknowledgement emails will only appear here if they are set to be processed by the automation service as opposed to the event service, this is enabled using 'Create acknowledgement emails in the background using an Automation' found within Configuration > Email > General Settings.</p><p><br></p><p><strong><em>Note: The log for the creation of automated tickets will not show here, these will show in the 'schedule occurrence log' tab</em></strong></p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6ImE1ZjdmNTRmLWJmY2ItNDEyMC05M2Y1LWRhNGRjYmQxNGRiZCJ9.MiFQyboV3OSVXTvXkGeYtvNn6CmKoSPvgSsFW2J7FFA" class="fr-fic fr-fil fr-dib" width="1216" style="width: 1218px; height: 379.486px;" height="379"></p><p><strong><span style="font-size: 10pt;">Fig 19. Automation Log</span></strong></p><p><br></p><p>In the log preview you will see when the automation took place, what action was automated, the status of the automation and, if the automation failed, a preview of the error message. </p><p><br></p><p>Click into a log to see more detail.</p><p><br></p><p>The description of the log entry will tell you what automation has taken place, if a custom action is being automated using a workflow, the description of the entry will be the name of the action. </p><p><br></p><p>In the 'Request' tab you will see the payload for the automation, along with a link to the ticket the automation is running for. The payload will contain various information about the automation including the workflow the automation is running on and the step of this workflow that contains the automated action. You can use this information to identify the source of the automation. </p><p><br></p><p><strong><em>Note: If a runbook is processing the automation the runbook ID will appear here too. </em></strong></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjIwM2NlMTA3LTg0ZjgtNDc3Ni1iMGQ0LWJjMTQ5ZjEyNGNjMiJ9.erqBKWClmMjPeXrEOl47QVK862CpIWCPbpPdf4NGtxc" class="fr-fic fr-fil fr-dib" width="771" style="width: 773px; height: 547.362px;" height="547"></p><p><strong><span style="font-size: 10pt;">Fig 20 . Request tab of workflow automation log</span></strong></p><p><br></p><p>In the 'Log' tab you can see how the automation has been processed, if the automation fails an error message will appear here. In the figure 21 example, we can see an action has been added to this ticket as action with ID = 31. </p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6ImE3NDJmNjFiLTM4MjAtNGE0Ni04OGU3LTZjMDk2M2IyNzFhZSJ9.Qq89y7ZFTRVQ1h18IUkS9PmOmBBWe5V9i7dSr7DbOY8" class="fr-fic fr-fil fr-dib" width="704" height="196"></p><p><strong><span style="font-size: 10pt;">Fig 21. Log tab of workflow automation log</span></strong></p><p><br></p><p>To find this action on the ticket head to the ticket > toggle the feed view to show all actions > edit an action, in the top right corner you will see the ID of the action. </p><p><br></p><p>You can use these logs to see if any automations have failed and identify the source of automations. If an automation has failed an error message will appear in the 'Log' tab, this can be used to troubleshoot issues with automations. </p><p><br></p><p><strong><span style="font-size: 12pt;">Schedule Occurrence Log</span></strong></p><p>Here, you can see logs for entities set to create on a schedule , this includes scheduled tickets and scheduled reports. Logs for the creation of automated tickets also show here. </p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjBkMGIxZmZiLTdlMmItNDhjNi1iZjBkLThlMWI4MWRiNjVmYiJ9.W-N0X9-1oy4Zu7HCSxcdDQYxasgcGd5N-KFgQfK1HpY" class="fr-fic fr-fil fr-dib" width="1220" style="width: 1222px; height: 330.753px;" height="331"></p><p><strong><span style="font-size: 10pt;">Fig 22. Schedule occurrence log tab</span></strong></p><p><br></p><p>In the log entries preview you can see a description for the entity that has been created, the date of creation and the status. </p><p><br></p><p>Click into a log entry to see more detail. </p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjI0ODRkNzczLWIyNTAtNGU3ZS1hMGQ4LTZkYjE3YjE4ODJhZiJ9.pEZ-fHmtvgu0ApzQpk4zTdrIA0pfppUvAokhKWxgFvY" class="fr-fic fr-fil fr-dib" width="878" height="555"></p><p><strong><span style="font-size: 10pt;">Fig 23. Automated ticket creation log </span></strong></p><p><br></p><p>In the request tab of the log there will be a link to the source ticket/report that the ticket/report has been created from. </p><p><br></p><p>The log tab of the ticket will contain the ID of the ticket that has been created from this schedule or any error messages if the creation failed. If the log is for a scheduled report the ID of the email that sent out containing the report will show here. This ID can then be searched in the 'ID' column of the outgoing log to see the details of the email that was sent. </p><p><br></p><p><strong><span style="font-size: 12pt;">Bulk Email</span></strong></p><p>Here, you can see logs for all bulk emails that have been sent out as part of a mail campaign. </p><p><br></p><p><strong><em>Note: If you are sending bulk emails using distribution lists, logs for these emails will not appear here, they will appear under the 'Outgoing Log' tab (an entry per email). </em></strong></p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjM3OTY0MGJiLTVjNGItNDRlYi04OTgxLTk0ZGIzMGY3ODQyZSJ9.ZVzGrkxhEsH258rHvzbEo_d7t9jXWAFCPdiHEhS34co" class="fr-fic fr-fil fr-dib" width="988" height="391"></p><p><strong><span style="font-size: 10pt;">Fig 24. Bulk email log</span></strong></p><p><br></p><p>The 'Request' tab of the log will contain a link to the mail campaign the bulk email was created by, as well as the mailbox the bulk email was sent from and the reply address for this bulk email. </p><p><br></p><p>The 'Log' tab will contain the number of total recipients that were set to be sent this mail, how many recipients actually received the mail and how many emails failed. </p><p><br></p><p>Each (individual) mail that is sent as part of a mail campaign will have a log in the 'Outgoing Log' tab, the log here will give you more information on why the email has failed. </p><p><br></p><p>You can use the logs here to identify if a whole, or part of, a bulk mail has failed. If failed mails are identified within this campaign you can then find these mails in the outgoing log to obtain information on why each mail has failed. </p><p><br></p><p><strong><span style="font-size: 12pt;">Monitoring events </span></strong></p><p>Here you can see errors and warnings that have occurred in the backend services (excluding integration errors). The events that show here will be based on the severity level you have set for error notification settings, the higher the level the more error notifications/warnings you will receive. </p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjdiYTgzNTUzLTc3NzAtNDdkNi1hMGZiLWRjNTZkMzNmZjQ5NCJ9.nrMpICRLCCm1UJ2ONo5AV0f0NAvAd8CogfkIT44t4o8" class="fr-fic fr-fil fr-dib" width="1222" style="width: 1224px; height: 351.235px;" height="351"><strong><span style="font-size: 10pt;">Fig 25. Monitoring events log</span></strong></p><p><br></p><p>Here you will see the time the error occurred, the severity of the error as well as a message indicating what the error is. </p><p><br></p><p>In the figure 25 example we can see a warning is being generated every couple of minutes to indicate 4 emails have failed to process. In order to obtain information on why each of these mails have failed to process we will need to find these mails in the 'Outgoing Log', this can be done easily by filtering the outgoing log to only show failed or pending mails. An entry will show here for every notification sent to alert you to errors. Notifications will keep firing until errors have been resolved, so although you may have many entries in the monitoring events tab, these will likely relate to the same error. </p><p><br></p><p>The monitoring events tab is used as the first point of call when investigating errors. When errors occur in the backend services you will be notified (in line with your notification settings), the notification will not contain too much detail on the error so you will need to navigate here to the monitoring events tab. Once here you can see what in the system has failed/errored and when the error started to occur. Once you know what is failing you can check the appropriate logs for more information on why the event has failed. </p><p><br></p><p>You will also see column for Log Type and Related Log ID.</p><p><br></p><ul data-pasted="true"><li>Log Type - this can be used to identify what kind of event it is and where the logs can be viewed (e.g. Task Scheduler or Incoming Email). </li><li>Related Log ID - this contains the ID of the most recent log that triggered the event.</li></ul><p><br></p><p><strong><span style="font-size: 12pt;">Integration Errors</span></strong></p><p>Here, you can see any errors that have occurred for integration syncs. When information is attempted to be imported/exported to a third party application and this fails, a log will show here.</p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjgyNmU1NDQzLTAzNmEtNDY4OC04NmZlLWZkNWM4MTYwNmQzNyJ9.sfKxn2MrzK3_95B92tWMQ5QfFZBRIfqlqgSbYdojdd4" class="fr-fic fr-fil fr-dib" width="1215" style="width: 1217px; height: 359.05px;" height="359"></p><p><strong><span style="font-size: 10pt;">Fig 26. Integration Errors log</span></strong></p><p><br></p><p>When previewing logs you will see the integration module the error is for, what entity has failed to sync and an error message. </p><p><br></p><p>If the error relates to a specific entity (such as a specific asset, client, user) clicking non the entry will take you to the profile of the entity. </p><p><br></p><p>The error message that displays here usually indicates why the sync has failed, however, you may need to check the Halo integrator or webhook (inbound/outbound requests) log for more information on the issue. </p><p><br></p><p>The monitoring events tab is used as the first point of call when investigating integration errors. When errors occur for integration syncs you will be notified (in line with your notification settings), the notification will not contain too much detail on the error so you will need to navigate here to the monitoring events tab. Once here you can see what in the system has failed/errored and when the error started to occur.</p><p><br></p><p><strong><span style="font-size: 12pt;">Integration Inbound Requests </span></strong></p><p>Here, you can see a log of all inbound requests from integration webhooks. Any integration that is syncing changes into Halo (inbound) using webhooks will create a log for each time they attempt a sync, here you will see a log for each successful/failed sync. </p><p><br></p><p><strong><em>Note: Syncs carried out by the halo integrator will not show here, these logs will show under the 'Halo integrator' tab. </em></strong></p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjRmNGQyNjA5LTdkZWMtNGQzZC05MjJjLTYwOWFkNzFkOGZiZiJ9.i21fX7mr3nrj3I3A8kZfcvCP9MCVe9XRBHgR38BKx4E" class="fr-fic fr-fil fr-dib" width="1217" style="width: 1219px; height: 465.231px;" height="465"></p><p><strong><span style="font-size: 10pt;">Fig 27. Integration inbound requests</span></strong></p><p><br></p><p>When previewing logs here you will see the integration module the log is for, the type of sync that occurred and when this occurred. </p><p><br></p><p>Click into a log to see more detail. </p><p><br></p><p>In the request tab we can see all the information about the incoming webhook. The URL tells us where the webhook has been sent from and the payload contains the payload of the incoming webhook. This will contain the fields that the webhook is attempting to update. </p><p><br></p><p>The response tab will contain the outcome/update that was made as a response to the webhook. If the request has failed to be processed an error message will appear here. </p><p><br></p><p><strong><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6ImJjN2Y0MWRjLTAwMTUtNDg0YS04Y2NkLWY3MzY2N2M2ZWYyNSJ9.0Jx5y6jfJ0lgiRoWoSaZYlSAXl6INkr6Y-QI_VWvZVg" class="fr-fic fr-fil fr-dib" width="557" height="255"></strong><br></p><p><strong><span style="font-size: 10pt;">Fig 28. Inbound integration response </span></strong></p><p><br></p><p><strong><span style="font-size: 12pt;">Integration Outbound Requests</span></strong></p><p id="isPasted">Here, you can see a log of all outbound requests from integration webhooks. This includes any posts Halo makes to an integration's API, either using webhooks or runbooks. Each outbound request will create a log, here you will see a log for each successful/failed sync.</p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjE5NTMxOGU2LTRkMGUtNDYzYi04ODBkLTRmZjJiZTNiNzZhMCJ9.fefvs3ap4RAM4RY0KbMSo3JngkDAU2m3maNRITur6Tc" class="fr-fic fr-fil fr-dib" width="1226" style="width: 1228px; height: 436.035px;" height="436"></p><p><strong><span style="font-size: 10pt;">Fig 29. Integration Outbound requests tab</span></strong></p><p><br></p><p>When previewing the log you will see the integration the log/request is for, the type of request that is being made (what entity is being updated) and when this request was made. </p><p><br></p><p>Click into a log to see more detail. </p><p><br></p><p>In the request tab you will see all the information on the outgoing webhook. The URL tells us where this webhook is being posted to and the payload section contains the payload of the webhook (the information that is being sent to the integration). </p><p><br></p><p>In the response tab you will see the outcome of the webhook, the body here will show you how the entity that is being updated has changed. If the request has failed to be processed the error will appear here. The figure 29 example shows the response of a webhook being posted to Xero to delete an invoice. Here we can see in the body the details of the invoice in Xero and how it has changed following the update from the webhook. </p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6IjcyNjcxZTRlLWVmNWUtNDA5YS1hNzc5LTA3Mjg4YmQzNDI4OCJ9.s9mwi7iqhe00z9iSQPgzl_QprDi8n2SADCtX-H2yHng" class="fr-fic fr-fil fr-dib" width="923" style="width: 925px; height: 700px;" height="700"></p><p><strong><span style="font-size: 10pt;">Fig 30a. Response following webhook to update invoice in Xero</span></strong></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6ImQ0MmQyYzk5LWY5NGMtNGVkNy04MmI1LTA4Mjc0ZDc3NTg1NyJ9.KVgtzKcU4kk16haQhF8gT1vdmsX62RfUAEikHOvE1XQ" class="fr-fic fr-fil fr-dib" width="919" style="width: 921px; height: 391.776px;" height="392"></p><p data-pasted="true"><strong><span style="font-size: 10pt;">Fig 30b. Section of response showing status of the invoice in Xero has been changed successfully</span></strong></p><p><br></p><p>This can be used to troubleshoot issues with outgoing webhooks/syncs, if an entity from an integration application is not being updated with information from Halo or is being updated incorrectly, you can come here to troubleshoot the issue. </p><p><br></p><p><br></p><p><strong data-pasted="true"><span style="font-size: 14pt;">Backend Service Monitoring for UAT Instances</span></strong></p><p>Backend Service Monitoring can be enabled with additional controls in UAT environments to prevent the overlap of services. From this version you will be able to enable the setting "Instance is a live UAT. Allow some automated backend processes" in the configuration tab of your Backend Service Monitoring. When enabled you will be able to "Enable outgoing emails and sms", and "Enable incoming emails". </p><p><br></p><p><strong><em>Note: To use these settings you must have separate mailboxes configured in each instance.</em></strong></p><p><br></p><p><img src="https://halo.haloservicedesk.com/api/attachment/image?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6ImUxODM4ZTY5LTkyYTAtNDEwOC1iN2Y3LTY5ZmNkMTc2YTFmYyJ9.KsXXvYXAYyIH_eis0dyl3tm-A-j53vR6JmSezSb2qcY" class="fr-fic fr-fil fr-dib" width="959" style="width: 961px; height: 470.604px;" height="471"></p><p data-pasted="true"><strong><span style="font-size: 10pt;">Fig 40. Backend Service Monitoring controls</span></strong></p><p><br></p><p><strong><span style="font-size: 14pt;">Common Errors</span></strong></p><p>For a breakdown of some common errors that can be found in the logs here, along with their resolutions, see our guide <a data-fr-linked="true" href="https://usehalo.com/haloitsm/guides/2477/" id="isPasted" target="_blank" rel="noopener noreferrer">Backend Services: Common Errors, Causes and Resolutions</a>. </p><p data-pasted="true"><br></p>
