[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=”4.4.2″ width=”100%” module_alignment=”center” global_colors_info=”{}”][et_pb_column type=”4_4″ _builder_version=”4.4.2″ global_colors_info=”{}”][et_pb_divider color=”#dddddd” _builder_version=”4.4.2″ global_colors_info=”{}”][\/et_pb_divider][\/et_pb_column][\/et_pb_row][\/et_pb_section][et_pb_section fb_built=”1″ custom_padding_last_edited=”on|phone” admin_label=”section” _builder_version=”4.4.2″ custom_margin_tablet=”” custom_margin_phone=”0px||0px||false|false” custom_margin_last_edited=”on|phone” custom_padding=”0px||||false|false” custom_padding_tablet=”” custom_padding_phone=”” transparent_background=”off” make_fullwidth=”off” use_custom_width=”off” width_unit=”off” custom_width_px=”1080px” custom_width_percent=”80%” global_colors_info=”{}”][et_pb_row column_structure=”2_3,1_3″ admin_label=”row” _builder_version=”4.4.2″ background_size=”initial” background_position=”top_left” background_repeat=”repeat” width=”100%” width_tablet=”85%” width_phone=”85%” width_last_edited=”on|phone” module_alignment=”center” custom_margin=”30px||||false|false” make_fullwidth=”off” use_custom_width=”off” width_unit=”off” custom_width_px=”1080px” custom_width_percent=”80%” global_colors_info=”{}”][et_pb_column type=”2_3″ _builder_version=”3.25″ custom_padding=”|||” global_colors_info=”{}” custom_padding__hover=”|||”][et_pb_post_title author=”off” comments=”off” featured_image=”off” title_tablet=”on” title_phone=”on” title_last_edited=”on|phone” meta_last_edited=”off|tablet” author_tablet=”off” author_phone=”off” author_last_edited=”on|desktop” date_tablet=”on” date_phone=”on” date_last_edited=”on|phone” categories_tablet=”” categories_phone=”” categories_last_edited=”on|phone” featured_image_tablet=”” featured_image_phone=”” featured_image_last_edited=”on|phone” disabled_on=”off|off|off” admin_label=”Post Title” _builder_version=”4.4.2″ title_font=”Open Sans|600|||||||” title_text_align=”left” title_text_color=”#000000″ title_font_size=”42px” title_letter_spacing=”-1px” title_line_height=”1.2em” meta_font=”Open Sans|600|||||||” meta_text_color=”#828282″ meta_font_size=”15px” meta_letter_spacing=”2px” meta_line_height=”3em” background_color=”rgba(0,0,0,0)” parallax=”on” parallax_method=”off” width=”85%” width_tablet=”” width_phone=”100%” width_last_edited=”on|phone” module_alignment=”center” custom_margin=”0px||20px||false|false” custom_margin_tablet=”” custom_margin_phone=”” custom_margin_last_edited=”on|desktop” custom_padding=”0px||0px||false|false” title_font_size_tablet=”38px” title_font_size_phone=”30px” title_font_size_last_edited=”on|phone” meta_font_size_tablet=”” meta_font_size_phone=”13px” meta_font_size_last_edited=”on|phone” meta_letter_spacing_tablet=”” meta_letter_spacing_phone=”1px” meta_letter_spacing_last_edited=”on|desktop” meta_line_height_tablet=”” meta_line_height_phone=”2em” meta_line_height_last_edited=”on|desktop” use_border_color=”off” border_color=”#ffffff” border_width=”1px” border_style=”solid” parallax_effect=”on” module_bg_color=”rgba(255,255,255,0)” global_colors_info=”{}” author__hover_enabled=”on|desktop” author__hover=”off”] [\/et_pb_post_title][et_pb_divider color=”#dddddd” _builder_version=”4.4.2″ width=”85%” width_tablet=”” width_phone=”100%” width_last_edited=”on|desktop” module_alignment=”center” custom_margin=”0px||0px||false|false” custom_padding=”10px||35px||false|false” global_colors_info=”{}”][\/et_pb_divider][et_pb_text admin_label=”Text: Intro Paragraph” _builder_version=”4.4.2″ text_font=”Abhaya Libre||||||||” text_font_size=”25px” text_line_height=”1.2em” background_color=”rgba(0,0,0,0)” width=”85%” width_tablet=”” width_phone=”100%” width_last_edited=”on|desktop” module_alignment=”center” text_font_size_tablet=”24px” text_font_size_phone=”23px” text_font_size_last_edited=”on|phone” global_colors_info=”{}”]<\/p>\n
While data automation is at it\u2019s best when you don\u2019t have to think about it, occasionally you\u2019ll want to see what\u2019s going on. One of the most common activities performed in Integrator is data analysis, especially for change data.<\/span><\/p>\n [\/et_pb_text][et_pb_text admin_label=”Text: Main Copy” _builder_version=”4.14.7″ text_font=”Open Sans||||||||” text_text_color=”#000000″ text_font_size=”17px” text_line_height=”1.6em” header_font=”Open Sans Light|on|||” header_font_size=”36px” header_2_font=”Open Sans|600|||||||” header_2_font_size=”23px” header_2_letter_spacing=”-1px” header_2_line_height=”1.5em” header_5_font=”Abhaya Libre||||||||” header_5_text_color=”#1464f6″ header_5_font_size=”30px” background_size=”initial” background_position=”top_left” background_repeat=”repeat” width=”85%” width_tablet=”” width_phone=”100%” width_last_edited=”on|phone” module_alignment=”center” custom_margin=”|||” custom_margin_tablet=”” custom_margin_phone=”” custom_margin_last_edited=”on|desktop” custom_padding_tablet=”” custom_padding_phone=”” custom_padding_last_edited=”on|phone” hover_enabled=”0″ text_font_size_tablet=”” text_font_size_phone=”18px” text_font_size_last_edited=”on|phone” text_line_height_tablet=”1.5em” text_line_height_phone=”1.6em” text_line_height_last_edited=”on|desktop” header_2_font_size_tablet=”” header_2_font_size_phone=”” header_2_font_size_last_edited=”on|phone” header_5_font_size_tablet=”” header_5_font_size_phone=”” header_5_font_size_last_edited=”on|phone” use_border_color=”off” border_style=”solid” global_colors_info=”{}” sticky_enabled=”0″]<\/p>\n <\/p>\n The most common data analysis scenario in Integrator<\/em> is when you are reacting to a Safeguard hold placed on a recent job due to the level of changes found. Other reasons include user inquiry about data in their apps; verifying agent configuration changes are working; data quality audit; or just occasional monitoring to see what\u2019s been happening in the data. No matter what the reason is, your path to analysis is the same and it will often take just a few minutes to conduct.<\/p>\n <\/p>\n Most of the time, you are looking at the data from the last job to answer a specific question and this is how it\u2019s done.<\/p>\n 1. Login to your Integrator<\/em> portal at https:\/\/dis.readwritedigital.com\/<\/a>. You will immediately see your \u201cJobs\u201d dashboard listing your agents and their most recent jobs. You may have one agent or one-hundred but you\u2019ll generally see something like the following list of 4 agents and their associated job data:<\/p>\n <\/p>\n 2. Each row in the table represents an agent with it\u2019s latest job status. Clicking on the blue text in the \u201cJob Status\u201d column will first display all of the data extracted and transformed for the job like the following example:<\/p>\n <\/p>\n 3. Let\u2019s say (using the above screenshot) you want to see data where the short_name of the data includes the text \u201cela\u201d. To do this, you\u2019ll add a filter. To filter the table on short_name = \u201cela\u201d, you can click in the text entry field under the short_name column header and type \u201cela\u201d and the list will instantly filter like:<\/p>\n <\/p>\n 4. Deleting \u201cela\u201d will bring back all the data again. You can filter like this on one or many columns in the job data view at the same time so you can narrow down to only the data you are interested in. By default a filter executes a \u201cContains\u201d search meaning Integrator will return any row for where that column contains the text you are filtering on – casting the widest net by default.<\/p>\n You can get more specific by clicking on the filter button in the column header Here is an example of directing Integrator to filter for only those rows where the short_name Equals \u201cela\u201d as opposed to simply containing that text.<\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n 5. In many cases, you will be most interested in the changed data so you\u2019ll want to click on the \u201cChange Data\u201d tab.<\/p>\n <\/p>\n <\/p>\n The main differences between All Data and Change Data are:<\/p>\n For \u201cupdate\u201d records in \u201cChange Data\u201d, the specific data that changed will be colored in red font<\/span> and if you hover your mouse over the red data<\/span>, you will see the previous value.<\/p>\n <\/p>\n <\/p>\n Looking at the screenshot of the Jobs dashboard again you\u2019ll see the fourth agent listed has \u201cClever (4)\u201d in the agent name column.<\/p>\n <\/p>\n A few things to notice:<\/p>\n If you click on the chevron for a complex agent, you will see something like the following:<\/p>\n <\/p>\n Here the \u201cParent\u201d agent is \u201cClever\u201d and it\u2019s children are \u201cClever Students\u201d, \u201cClever Teachers\u201d and so forth. In the \u201cJob Status\u201d column this agent indicates \u201cSAFEGUARD ACTIVATED\u201d for the Parent and one child, \u201cClever Sections\u201d \u2026 and you can see why!<\/p>\n Each child has different data and clicking on its link under \u201cJob Status\u201d allows you to see the data for that particular child. While the parent agent will have a status indicated, the parent often has no data associated with it specifically so clicking on it\u2019s link may show you an empty data grid. Essentially, an agent will only have data if it is configured (under \u201cAgents\u201d) such that the \u201cExtract data format\u201d setting is not \u201cNone\u2026\u201d or \u201cCompression archive\u2026\u201d.<\/p>\n <\/p>\n Child agents will often be related via common key such as an ID value present in more than 1 agent. Sometimes you\u2019ll want to open the data for the child agents in separate windows so you can filter and look at related data all at once.<\/span><\/p>\n Complex agents work best when their jobs are executed at the parent level. On the \u201cJobs\u201d dashboard, If you see child jobs with extract start times \u201cout of sync\u201d with the parent extract start time \u2013 a child started before the parent or way after all other children and the parent \u2013 you are probably dealing with an \u201cout of sync\u201d job. This can sometimes happen if a user manually starts or rolls back a child job specifically, instead of doing it at the parent level. As a rule of thumb, users should always \u201cExecute\u201d manual commands on a complex job at the parent level unless they have a good reason not to. For out of sync jobs, it may be difficult to come to very certain conclusions about the overall job during your data analysis.<\/p>\n <\/p>\n The \u201cJobs\u201d dashboard includes a graph of data changes over time. Which you can see by clicking on the chevron located in the top right corner of the page\u00a0\u00a0 <\/p>\n In the example above, the \u201cData changes period\u201d menu is set to \u201cAll Time\u201d selected but we see the change data looks really flat for most of the jobs because we have some huge spikes in the Clever change data. Often that sort of change can be traced to year end transitions or even manual manipulations of the job that artificially caused a spike in changes for a job or two. Clicking the \u201cHide outlier data\u201d checkbox at the top of the chart will rationalize the data bit.<\/p>\n <\/p>\n If we are really just interested in the activity of one agent, click the agent names in the legend on the right until the one you want shows up. Hovering over the graph indicates specific change counts for specific timeframes. Like the example below.<\/p>\n<\/h2>\n
![\"\"](\"https:\/\/www.readwritedigital.com\/wp-content\/uploads\/2020\/08\/0-intro-job-list-student-row.jpg\")
<\/a><\/h2>\n<\/h2>\n
Why analyze job data?<\/h2>\n
Analyzing job data: Step-by-Step<\/h2>\n
<\/a><\/p>\n<\/a><\/p>\n<\/a><\/p>\n<\/a>\u00a0and select one of the filter \u201cqualifiers\u201d if you don\u2019t want the default \u201cContains\u201d search.<\/p>\n<\/p>\n<\/a><\/p>\n\n
\n
<\/a><\/p>\n<\/h2>\n
Complex agents: parent \/ child relationships<\/h2>\n
<\/a><\/p>\n\n
<\/a><\/p>\n<\/a><\/p>\n<\/a><\/p>\nThe big picture: looking at data change rates over time<\/h2>\n
<\/a> next to the job list and just under the \u201cAccount\u201d menu item.<\/p>\n<\/a><\/p>\n<\/a><\/p>\n