Analytics reports: Extraction

The Extraction menu contains reports that depict how well AP Essentials automatically finds fields on documents. Use the Account type box to filter accounts by accounts payable and accounts receivable. Success rates are determined by analyzing how often users change field values. Generally, if extracted field values are not being changed by users during verification, this is an indication that AP Essentials is extracting the fields correctly. By examining the ratios between fields which AP Essentials thinks are Complete, with fields that are changed and unchanged by users, we can determine extraction performance. For example, if a particular field value is never changed, it means AP Essentials correctly found the field 100% of the time. As a result, this field would have a high rate of success (100%). However, if a particular field value is changed frequently, it means AP Essentials has difficulty finding the field, and it would have a low success rate.

Extraction data is calculated using documents that were extracted and manually verified during the time you specify in the Date range. The data can include documents that are still being processed in AP Essentials. Invoices that skip manual verification are not included in the extraction statistics. For example, if 100 documents were extracted during the Date range, and 80 documents skipped manual verification, only the 20 documents that were manually verified will be included in the extraction statistics.

Note that solutions that use external extraction may not provide any extraction statistics because the API does not require feedback of external verification data.

Success rates for suppliers and buyers are measured in terms of identification rates, confidence rates and correctness rates. When calculating rates for fields, for example, we analyze the relationships depicted in the diagram below.

Note that the green area in the middle depicts an overlap of fields that are marked as Complete by AP Essentials and fields that were not changed by users during verification.

Success rates for suppliers and buyers are measured in terms of:

• Identification rate - Measures how successfully suppliers or buyers are automatically identified. The identification rate is determined by comparing the number of unchanged fields to the total number of fields (changed and unchanged). The rate is calculated by dividing the number of unchanged fields with the total number of fields and multiplying by 100 to get the percentage.

  For example, assume you are looking at the supplier identification rate of 100 documents, where 90 supplier fields were not changed during verification. In other words, there are 100 supplier fields in total, 10 of which were changed during verification. The other 90 were not changed during verification.

  (90/100) × 100 = 90%

  The result is an identification rate of 90%, which means 90% of the suppliers were automatically identified correctly.

• Confidence rate - Measures how much users can rely on AP Essentials to correctly find supplier or buyer fields. The confidence rate is similar to the identification rate, but the pool of fields is limited to fields that AP Essentials thinks it captured correctly. The confidence rate answers the question: what percentage of fields that AP Essentials thought it captured correctly were actually captured correctly? The confidence rate is calculated by dividing the number of fields that AP Essentials assigned the Complete status to (and the user did not change during verification), with the number of fields that AP Essentials assigned the Complete status to.

  For example, assume you are looking at the supplier confidence rate of 100 documents. AP Essentials assigned the Complete status to 95 supplier fields. Of those 95 Complete fields, only 5 were changed during verification. In other words, AP Essentials and the users agree that 90 of the fields were captured correctly.

  (90/95) × 100 = 94.7%

  The result is a confidence rate of 94.7%, which means users can rely on AP Essentials to correctly find supplier fields 94.7% of the time.

• Correctness rate - Measures the percentage of fields that are automatically extracted correctly without user interaction. The correctness rate is determined by comparing the number of fields that AP Essentials assigned the Complete status, which were also unchanged during verification, to the total number of fields (changed and unchanged). The rate is calculated by dividing the number of Complete and unchanged fields with the total number of fields and multiplying by 100 to get the percentage.

  For example, assume you are looking at the supplier confidence rate of 100 documents. AP Essentials assigned the Complete status to 95 supplier fields. Of those 95 Complete fields, only 5 were changed during verification. In other words, AP Essentials and the users agree that 90 of the fields were captured correctly.

  (90/100) × 100 = 90%

  The result is a correctness rate of 90%, which means supplier fields do not require user interaction 90% of the time.

The success rates for other fields are measured in terms of:

• True positive - AP Essentials captured the field correctly and assigned the Complete status to the field. The user did not change the field value during verification. In this case, AP Essentials and the user agree the field was captured correctly.
• False positive - AP Essentials thinks it captured the field correctly and assigned the Complete status to the field. However, the user changed the field value during verification. In this case, the user determined that AP Essentials did not capture the field correctly.
• True negative - AP Essentials had trouble finding the field and assigned the Uncertain status to the field. The user changed the field value during verification. In this case, AP Essentials and the user agree that the field was not captured correctly.
• False negative - AP Essentials had trouble finding the field and assigned the Uncertain status to the field. However, the user did not change the field value. In other words, the field was captured correctly, even though AP Essentials was not sure about its correctness. In this case, the user determined that AP Essentials did capture the uncertain field correctly.

These reports are available:

• Supplier identification and field extraction - Displays supplier identification and field extraction rates. Triangles indicate the percent change over the last 30 days.

• Supplier identification over time - Displays supplier identification rates over time, so you can see if rates are improving or worsening. This is helpful, for example, to see if changes made to supplier master data increase your identification rates. Triangles indicate the percent change over the last 30 days.

• Supplier identification by organization - Displays supplier identification rates over time, grouped by organization. Triangles indicate the percent change over the last 30 days.

• Buyer classification - Displays classification rates over time, grouped by organization. Triangles indicate the percent change over the last 30 days.