--- title: SFTP Data Dictionary excerpt: '' deprecated: false hidden: true metadata: title: '' description: '' robots: index next: description: '' --- # What is the daily data format? SFTP data shows every single resolution and interaction event, which will allow customers more flexibility in filtering/aggregating the data to build custom reports. **SFTP data covers usage of Classic Moveworks bot and does not include data on usage of Copilot.** This document provides information on the data dictionary (the tables and fields involved), and directions on how it can be aggregated to build the views shown on your daily dashboard that Moveworks provides in Tableau. With this knowledge, you'll have the power to take the data and use it in any internal views/reports/dashboards you may need to build. # Data Dictionary The daily SFTP data format contains multiple files, while some are used more extensively than others to derive the key metrics, all are necessary to build the complete daily dashboard that Moveworks provides. The files are as follows, and are all prefixed with today's date: * \{\{today's date}}\_daily\_detailed\_report.csv * \{\{today's date}}\_daily\_active\_users.csv * \{\{today's date}}\_daily\_concierge\_tickets.csv * \{\{today's date}}\_daily\_user\_roster.csv * \{\{today's date}}\_daily\_triage\_record.csv * \{\{today's date}}\_daily\_answers\_insights\_user\_interactions.csv * \{\{today's date}}\_daily\_new\_tickets.csv The date format will be as follows: year-month-day e.g: `2023-12-13_daily_active_users.csv ` ## File 1: Resolution and Interaction Events File name: `{{today's date}}_daily_detailed_report.csv` This is the main file that contains a table of all of the resolution and significant interaction events for the day. Each row in this table represents a resolution or interaction. Below are the descriptions of each column in the table:
Column Description
timestamp\_utc Time stamp in UTC (Coordinated Universal Time). Quick conversion tips: PST is 7 hours earlier than UTC time; EST is 4 hours earlier than UTC.
email\_address Email address of the user who experienced the resolution or interaction.
initiator The value in this column delineates whether it was the User or the Bot that initiated the conversation that led to this event. Thus, the only possible values are 'User' and 'Bot'.
skill\_category The high-level skill category for this event. The possible values are 'Access Software', 'Access Group', 'Access Account', 'AskMe', 'Lookups', 'Form Filing', and 'Interaction'.
skill\_detail More precise information on the use case and/or entity involved in this resolution. The type of information provided here depends on the skill category (the previous column). If the skill\_category is 'Access Software', this column will contain the name of the app that was provisioned or for which self-service instructions were given to the user. If the skill\_category is 'Access Group', this column will contain an indicator of whether the resolution was a create operation ('Create'), an add operation ('Add to'), or remove operation ('Remove from'). If the skill\_category is 'Access Account', this column will contain the precise use case. The possible values are: 'Change PW prompted' (upon a password expiration reminder), 'Change PW unprompted' (the user asking to reset their password), 'Unlock Account', 'Reset MFA', or 'Account Expiry' (if the user took action on a contractor/contingent worker account expiry reminder) If the skill\_category is 'AskMe', there are two possible values for this column: 'AskMe answer Served' (if the bot provided a possible answer to a user's question and the user did not rate it as "not helpful") or 'AskMe rated Not Helpful' (if the bot provided a possible answer, but the user clicked the button to say it was not helpful or filed a ticket) If the skill\_category is 'Form Filing', this column will contain the name of the catalog item that the user submitted If the skill\_category is 'Lookups', there are several possible values for this column. If the lookup was considered successful (i.e. the user did not rate it as "not helpful"), the value of this column will correspond to the type of lookup it was: 'people', 'rooms', 'finance', or 'partner'. If the bot provided more than one potential matches, the column will be empty. If the skill\_category is 'Interaction', this column will contain the precise use case for the user's interaction with the ticketing system. The possible values are: 'Nudge Agent', 'Comment added', 'Ticket closed in bot', 'Ticket weblink clicked', 'Check Status', 'Click on Got it Thanks' (when provided an update on the ticket), 'Ticket submitted in bot', 'Tickets approved' (for ServiceNow RITM only), 'Live Chat clicked', 'Other interaction' (ticket reopening or interactions with tickets that were opened due to resolution events), 'Campaign response'.
ticket\_id This column contains the ID number of the ticket that is associated with this event, if one exists. A ticket is associated with an event in two cases: (1) if it is responsible for the bot initiating the conversation (i.e., the bot "reached out" to the user upon reading their ticket), or (2) if the bot files a ticket in the process of automatically resolving the user's issue (to create a log of the event for audit purposes).
user\_id\* Numeric ID of the user who experienced the resolution or interaction, generated internally by Moveworks
minutes\_to\_resolution\* Time between when an issue was created and when it was resolved in the bot, in minutes
domain\* Domain of enterprise service related to the issue described, as detected by ML models per user utterance
issue\_origin\* Origin of issue indicates where user described their issue. It can be user utterance in the bot chat or in IT chat channel, or ticket intercepted by bot.
\*available in some versions of SFTP data ## File 2: List of active users File name: `{{today's date}}_daily_active_users.csv` This file contains a table that shows all of the users who were active during the day. A user is considered to have been "active" if they sent a message to the bot OR if they clicked a button/link in a message from the bot. Below are descriptions of each column in the table: | Column | Description | | :------------- | :------------------------------------------------- | | dt | The day's date in UTC (Coordinated Universal Time) | | email\_address | User's email address | | user\_id | Unique user ID generated internally by Moveworks | ## File 3: List of tickets that were updated in the ITSM system File name: `{{today's date}}_daily_concierge_tickets.csv` This file contains a table that shows the list of tickets that were available to the bot for Concierge use cases (eligible for checking status, receiving updates, adding comments, etc.). It is used in calculating the percentage of eligible tickets that users interacted with through the bot. Below are descriptions of each column in the table: | Column | Description | | :--------- | :------------------------------------------------------------------------------ | | dt | The day's date in UTC (Coordinated Universal Time) when ticket is polled by bot | | ticket\_id | ID of the ITSM ticket corresponding to the request | | domain | The domain of the issue described in ticket | ## File 4: Full roster of eligible bot users File name: `{{today's date}}_daily_user_roster.csv` This file contains a table that shows the full roster of users in your organization that are (or were previously) available to the Moveworks system. It is used to produce the breakdowns of active users by geographic location and department. Below are descriptions of each column in the table: | Column | Description | | :--------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | email\_address | The email address of the user | | geo | The user's geographic location, as stored in Moveworks' user roster. This comes from one of your systems of record, such as ServiceNow or Active Directory (you can ask your Customer Success Manager if the values don't look accurate) | | department | The user's department, as stored in Moveworks' user roster. This comes from one of your systems of record, such as ServiceNow or Active Directory (you can ask your Customer Success Manager if the values don't look accurate) | | country\_code | Country code of the user | | is\_active\_in\_roster | Whether the user is active in the roster. Either True or False, shows if the user is present in the current user identity roster that we pull from either your ITSM or AD roster | | has\_access\_to\_bot | Whether the user has access to the Moveworks Bot based on whitelisting. If is\_active\_in\_roster is False, then has\_access\_to\_bot will be False | | user\_id | Unique user ID generated internally by Moveworks | ## File 5: List of tickets triaged by the bot File name: `{{today's date}}_daily_triage_record.csv` This file contains a table that shows all the tickets triaged by the model (it will only be present if the Triage skill is active in your environment). If models are being used to set multiple ticket fields (e.g. Call Type, Assignment Group, Category, etc.), predictions and results for all fields will be in this single file. Below are descriptions of each column in the table:
Column Description
timestamp\_utc Time stamp in UTC (Coordinated Universal Time) when ticket is polled by bot
ticket\_id ID of the ITSM ticket corresponding to the request
predicted\_field The specific field that is being predicted by the model. These fields include but are not limited to assignment\_group, business\_service, category, subcategory, and components.
start\_value The field value when the ticket was CREATED (before the model makes a prediction)
predicted\_value The model prediction of the field value
final\_value The field value when the ticket was CLOSED. The final value can either match the predicted value or be updated by agents.
short\_description The short description of the ticket id
triage\_result This field is populated once the ticket is CLOSED. The type of information provided here depends on the predicted field: If the predicted field is "call type", a successful prediction would result in a ticket "Predicted" (incident, service request, etc), "No Prediction" (no prediction of field value), or "Out of Scope" (an agent updated the ticket before the bot).  For other predicted fields such as "Assignment Group", a prediction can be "Correct" or "Incorrect" (comparing predicted value to final value), "No Prediction" (no prediction of field value) or "Out of Scope" (an agent updated the ticket before the bot).
predicted\_field\_type This field is populated once the ticket is CREATED. The type of action the model is performing to the specific field. For CALL tickets, a model would transfer a ticket "Transferred". For other tickets, such as INC or ITHELP types, they can be "Routed" (routed the ticket to the appropriate support group), "Categorized" (labeled the ticket for reporting purposes), or "No Change" (predicted value is the same as the start value).
## File 6: List of User Interactions in scope of Answers The data in this file corresponds to the current edition of the Answers Insights customer-facing Tableau dashboard. File name: `{{today's date}}_daily_answers_insights_user_interactions.csv` This file contains a table that shows all issues (utterances and tickets) that have the potential to be resolved by Answers skill, indicating the outcome of the bot attempt to solve the issue and context around it. Data uniqueness is defined by the combination of the issue, knowledge article, and issue topic. For example, if 3 articles were served for an issue where 2 issue topics were detected, there will be 6 rows in the data. Below are descriptions of each column in the table:
Column Description
timestamp\_utc Timestamp (UTC) indicating when the question was asked in bot or when bot reached out with suggestion on a ticket
article\_added\_date Date when the knowledge article was added
user\_id Unique user ID generated internally by Moveworks
user\_name User name (will be redacted for Enterprise Answers)
department User department (will be redacted for Enterprise Answers)
language Language of the request (may be empty in case of feature not enabled)
user\_text Text of the user request or ticket
knowledge\_title Title of the knowledge snippet served to user
article\_id Knowledge article ID
doc\_source Source of the knowledge snippet (Internal Knowledge Base/Moveworks FAQ Sheet/Moveworks External Knowledge)
doc\_domain Domain of the knowledge article, as set when the article was added
issue\_topic Entity detected in user request (Email/Outlook/Zoom/VPN/...)
result Result of the bot actions on the request "Not Served": No solution is offered to the user, instead bot suggests user to hand the issue off to an agent.  "No Engagement: Bot is able to find a relevant answer, however user does not engage to see the suggestion. This can happen in Ticket route if the user does not click on any article links. Another case is when the bot suggests solutions from multiple skills but user does not choose to see results from Answers. "Not Rated": Bot offers an answer to the user; the user does not give any explicit feedback and does not hand the issue off to an agent. (Agent handoff includes filing a ticket, click on a Live Agent link or redirection to another domain portal.) "Agent Handoff": Bot offers an answer to the user but the user proceeds to an agent handoff. "Rated Negative": Bot offers an answer to the user but the user rates the answer as not helpful. "Rated Positive": Bot offers an answer to the user and the user rated the answer as helpful.
source Origin of the request "DM": direct message to bot "Ticket route": intercepted ticket "Group channel":message intercepted in a group channel
article\_url URL of the knowledge article served
ticket\_id ID of the ITSM ticket corresponding to the request
ticket\_url URL of the ITSM ticket corresponding to the request
## File 7: List of daily new tickets File name `{{today's date}}_daily_new_tickets.csv` This file contains the list of new tickets ingested daily from ITSM system. | Column | Description | | :------------- | :-------------------------------------------------------------------------------------------------------------------------------------------- | | timestamp\_utc | Time stamp in UTC (Coordinated Universal Time). Quick conversion tips: PST is 7 hours earlier than UTC time; EST is 4 hours earlier than UTC. | | ticket\_id | ID of the ITSM ticket corresponding to the request | | domain | Domain of the issue described in ticket | ## File 8: List of Channel Resolver issues File name `{{today's date}}_daily_channel_issues.csv` This file contains the list of issues in the Slack channels where Channel Resolver is active in. | Column | Description | | :---------------------- | :-------------------------------------------------- | | channel\_id | The ID of the Slack channel | | full\_name | The full name of the user who posted in the channel | | resolution\_skill\_type | The skill that the bot reached out to the user with | | issue\_feedback\_type | The user feedback on the bot reach-out | | is\_issue\_resolved | Resolution outcome of the issue | | dt | Timestamp | # Description of metrics derived from the data In the following section, we go through commonly used metrics shown in Performance Insights dashboards that Moveworks provides in Tableau, providing a description and the corresponding SQL query or arithmetic calculation you can use to derive these metrics from the data in the files discussed above. ## HIGH-LEVEL METRICS ### Absolute Number Metrics These metrics are counts of users, resolution, interactions, and forms filed. Below is a table that outlines how each of these metrics are calculated:
Metric Description SQL Query or Arithmetic Calculation
Active Users Count unique number of users (by email address) in the daily active users table select count(distinct email\_address) from \{\{date}}\_daily\_active\_users
Resolution Count all resolution events from the main events table, excluding Interactions and negative feedback (i.e. where the user rated the bot's response "not helpful). select count(\*)\ from \{\{date}}\_daily\_detailed\_report \ where skill\_category not in ( 'Interaction', 'Lookups')\ and skill\_detail not in ('AskMe rated Not Helpful')
Interactions Count the number of unique tickets that users interacted with through the bot, from the main events table select count(distinct ticket\_id)\ from\ \{\{date}}\_daily\_detailed\_report\ where skill\_category='Interaction'
### Percentage Metrics There are two percentage numbers in this section - resolution percentage and interactions percentage. The key to understanding these metrics is to understand what is included in the numerator of the percentage calculation and what is included in the denominator. **Resolution Percentages** This metric calculates a percentage of users' issues resolved by the bot. All relevant resolution events from the main events table are counted in the numerator (only events that are rated "not helpful" are excluded). To obtain the correct denominator, the starting point is the number of new tickets considered for resolution on the day (using SQL, this can be found with the following query: `select count(distinct ticket_it) from {{date}}_daily_new_tickets`). But this number is not the complete denominator, because many of users' issues are addressed through chatting with the bot and no corresponding ticket is filed. Thus, we need to add these events to the denominator, whether they were resolved or not. At a high level, the events that need to be added to the denominator are those that do not originate from a ticket, and for which no new ticket is filed. These can be found by choosing the events that do not have the ticket\_id column populated (using SQL, this can be found with the following query: `select count(*) from {{date}}_daily_detailed_report where skill_category <> 'Interaction' and ticket_id is null`). Since the logic behind whether a ticket is filed depends on the skill\_category, it is best to explain the details with the following table that looks at all the types of resolution events (i.e. combinations of skill\_category and skill\_detail): | skill\_category | skill\_detail | Included in Resolution? | Conditions for addition to denominator (i.e. no ticket filed) | | :-------------- | :----------------------------- | :---------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Access Account | Change PW prompted | Yes | If the conversation was initiated by the user and the bot provides instructions to the user or redirects them to a self-service password reset link without taking action on the identity system | | Access Account | Change PW unprompted | Yes | If the bot provides instructions to the user or redirects them to a self-service password change link, without taking action on the identity system | | Access Account | Reset MFA | Yes | If the conversation was initiated by the user and the bot provides instructions to the user or redirects them to a self-service MFA reset link, without taking action on the identity system | | Access Account | Unlock Account | Yes | If the bot provides instructions to the user or redirects them to a self-service account unlock link, without taking action on the identity system | | Access Groups | Add to | Yes | If the bot provides instructions to the user or redirects them to a self-service group management portal, without performing the addition automatically | | Access Groups | Create | Yes | If the bot provides instructions to the user or redirects them to a self-service group management portal, without performing the creation automatically | | Access Groups | Remove from | Yes | If the bot provides instructions to the user or redirects them to a self-service group management portal, without performing the creation automatically | | Access Software | *app name* | Yes | If the conversation was initiated by the user and the bot provides instructions to the user or redirects them to a self-service portal, without provisioning software automatically | | Askme | AskMe answer served, not rated | Yes | If the conversation was initiated by the user (not from an existing ticket) | | Askme | AskMe rated Helpful | Yes | If the conversation was initiated by the user (not from an existing ticket) | | Askme | AskMe rated Not Helpful | No | If the conversation was initiated by the user (not from an existing ticket) | | Form Filing | form name | Yes | Since there is always a ticket associated with a form, no addition is ever necessary | **Interactions Percentage** The goal here is to calculate the percentage of tickets that users interacted with through the bot. Thus, the numerator is the count of unique tickets that had interaction events in the main events table (using SQL, the query would be `select count(distinct ticket_id) from {{date}}_daily_detailed_report where skill_category='Interaction'`). The denominator is the total count of unique tickets that were eligible for Concierge interactions that day, from the daily concierge tickets table (using SQL, the query would be `select count(distinct ticket_id) from {{date}}_daily_concierge_tickets`). ## EVENTS BREAKDOWN The table below provides descriptions and SQL queries for the Resolution breakdown.
Type of breakdown Description SQL Query
Resolution by skill Count rows in the main event table by respective resolution skill\_category, excluding Interactions and negative feedback (i.e. where the user rated the bot's response "not helpful). select skill\_category, count(\*) from \{\{date}}\_daily\_detailed\_report where skill\_category not in ( 'Interaction', 'Lookups') and skill\_detail\<>'AskMe rated Not Helpful' order by count desc
Access Group by type of operation Count Access Software resolutions by software application select SPLIT\_PART(skill\_detail, ' ', 1) as type, count(\*) from \{\{date}}\_daily\_detailed\_report where skill\_category = 'Access Group' group by type
Access Account by use case Count Access Account resolutions by use case select skill\_detail, count(\*) from \{\{date}}\_daily\_detailed\_report where skill\_category = 'Access Account' group by skill\_detail
Access Software by App Count Access Software resolutions by software application select skill\_detail, count(\*) from \{\{date}}\_daily\_detailed\_report where skill\_category = 'Access Software' group by skill\_detail
AskMe by feedback Count AskMe events by those resolved ('AskMe answer served') and those not resolved ('AskMe rated Not Helpful') select skill\_detail, count(\*) from \{\{date}}\_daily\_detailed\_report where skill\_category = 'AskMe' group by skill\_detail
Forms filed Count the number of forms filed, grouping by the catalog item name. select skill\_detail, count(\*)\ from \{\{date}}\_daily\_detailed\_report\ where skill\_category='Form Filing' group by skill\_detail
Lookups by feedback Count successful Lookup events by type of lookup select skill\_detail, count(\*) from \{\{date}}\_daily\_detailed\_report where skill\_category = 'Lookups' and skill\_detail \<> 'Lookup rated Not Helpful' group by skill\_detail
Interactions by type Count the interaction events in the main events table, grouping by the type of interaction. It is a simple count for all types of interaction. select skill\_detail, count(\*)\ from \{\{date}}\_daily\_detailed\_report\ where skill\_category ='Interaction' group by skill\_detail
## ACTIVE USERS The table below provides descriptions and SQL queries to count number of users engaged with Moveworks bot.
Metric Description SQL Query
Number of active users by department Count unique users (by email address), grouping by department, after joining the active user table to the user roster table select \{\{date}}\_daily\_user\_roster.department, count(distinct \{\{date}}\_daily\_active\_users.email\_address) from \{\{date}}\_daily\_active\_users join \{\{date}}\_daily\_user\_roster on \{\{date}}\_daily\_active\_users.email\_address=\{\{date}}\_daily\_user\_roster.email\_address group by \{\{date}}\_daily\_user\_roster.department
Number of active users by geo Count unique users (by email address), grouping by geographic location, after joining the active user table to the user roster table select \{\{date}}\_daily\_user\_roster.geo, count(distinct \{\{date}}\_daily\_active\_users.email\_address) from \{\{date}}\_daily\_active\_users join \{\{date}}\_daily\_user\_roster on \{\{date}}\_daily\_active\_users.email\_address=\{\{date}}\_daily\_user\_roster.email\_address group by \{\{date}}\_daily\_user\_roster.geo
## TRIAGE PERFORMANCE The table below provides descriptions and SQL queries for Triage metrics.
Metric Description SQL Query
Triage Result by Field Count the number of triage\_result\_type, grouped by the predicted\_field and triage\_result. select predicted\_field, triage\_result, count(\*) from \{\{date}}\_daily\_triage\_record group by predicted\_field, triage\_result
Breakdown of Triage Result by Predicted Values Count the number of predicted\_value, grouped by the predicted\_field, triage\_result, and predicted\_value. select predicted\_field, triage\_result, predicted\_value, count(\*) from \{\{date}}\_daily\_triage\_record group by predicted\_field, triage\_result, predicted\_value
## KNOWLEDGE The table below provides descriptions and SQL queries for Knowledge metrics.
Metric Description SQL Query
Articles Served Total number of times an article was found select count(distinct article\_id||timestamp) from \{\{date}}\_answers\_insights\_user\_interactions where result in ('Rated Positive', 'Not Rated', 'Rated Negative', 'Agent Handoff', 'No Engagement')
Answers Provided Total number of issues when a solution was suggested to user select count(distinct user\_text||timestamp) from \{\{date}}\_answers\_insights\_user\_interactions where result in ('Rated Positive', 'Not Rated', 'Rated Negative', 'Agent Handoff')
Resolutions Provided Total number of issues when a solution was suggested to user and there is no negative feedback or a hand-off to an agent * In Channel route, only issues with positive feedback are counted select count(distinct user\_id||timestamp) from \{\{date}}\_answers\_insights\_user\_interactions\ where\ source='Group channel' and result='Rated Positive'\ or\ source='Ticket route' and (result='Rated Positive' or result='Not Rated')\ or\ source='DM' and (result='Rated Positive' or result='Not Rated')
Employees with Resolutions Number of unique users with a solution suggested and no negative feedback or a hand-off to an agent * If a user reported an issue in Channel route, only resolutions with positive feedback are taken into account select count(distinct user\_id) from \{\{date}}\_answers\_insights\_user\_interactions\ where\ source='Group channel' and result='Rated Positive'\ or\ source='Ticket route' and (result='Rated Positive' or result='Not Rated')\ or\ source='DM' and (result='Rated Positive' or result='Not Rated')
Knowledge Effectiveness by Article Ratio of the number of issues with resolution in Answers (not rated or rated positive) to the number of issues with Answers suggestion served to user select knowledge\_title, 1.0\*count(distinct case when result not in ('Rated Negative', 'Agent Handoff', 'Not Served', 'No Engagement') then timestamp else null end)/count(distinct case when result not in ('Not Served', 'No Engagement') then timestamp else null end) as knowledge\_effectiveness\_rate from \{\{date}}\_answers\_insights\_user\_interactions group by knowledge\_title
Knowledge Coverage by Issue Topic Ratio of the number of issues with a knowledge article found to the number of issues with an attempt to find an article select issue\_topic, 1.0\*count(distinct case when result not in ('Not Served') then timestamp else null end)/count(distinct timestamp) as knowledge\_coverage\_rate from \{\{date}}\_answers\_insights\_user\_interactions group by issue\_topic
# FAQ **Q: When can I expect SFTP data to land?** **A:** SFTP reports will be produced the following day by **11:00 PM UTC Time**. For example, the data for the 24 hours of **January 1st** will land by **11PM UTC on January 2nd.** **Note:** data from each file will have no overlap of data. **Q: Do the files grow in size?** **A:** No, the files are sent for the days data, and a new file will be added for the next day.