Difference between revisions of "Troubleshooting NLP"

From PegaWiki
Jump to navigation Jump to search
(updated application name and industry)
Tag: Visual edit
(Curated the article.)
Tag: Visual edit
 
Line 1: Line 1:
 
{{Design pattern|Title=Troubleshooting NLP|Description=Troubleshooting NLP issues - so that customers can self service NLP issues before reaching GCS.|Version=8.2|Applications=Pega Platform|Capability Area=Conversational Channels|Owner=Gudis1}}
 
{{Design pattern|Title=Troubleshooting NLP|Description=Troubleshooting NLP issues - so that customers can self service NLP issues before reaching GCS.|Version=8.2|Applications=Pega Platform|Capability Area=Conversational Channels|Owner=Gudis1}}
  
== How to troubleshoot NLP issues with email, chat and messaging==
+
== How to troubleshoot NLP issues with email, chat, and messaging ==
Text analytics or NLP (Natural language processing) issues fall under Text categorization (Topic, Small talk, Sentiment and Language) and Text extraction (Entities, keywords and auto tags). Most of the issues can be handled in the channel interface using Dev Studio. However, the advanced issues may require access to Prediction Studio. This article mainly covers the major NLP issues that can be addressed using the channel interface itself.   
+
Text analytics and natural language processing (NLP) issues are related to either text categorization (topic, small talk, sentiment, and language) or text extraction (entities, keywords, and auto tags). You can handle most of the issues in the channel interface by using Dev Studio. However, advanced issues might also require access to Prediction Studio. Information presented below covers the majority of NLP issues that you can address by just using the channel interface in Dev Studio.   
  
 
== Symptoms of a potential NLP issue ==
 
== Symptoms of a potential NLP issue ==
  
==== In Email ====
+
==== Email ====
 +
The following NLP issues relate to email:
 
* Topic is not detected
 
* Topic is not detected
 
* Topic is detected but the case is not assigned
 
* Topic is detected but the case is not assigned
Line 16: Line 17:
 
* Signature and greetings in email body part of entities detected
 
* Signature and greetings in email body part of entities detected
  
===== In Chat, Messaging and Live chat =====
+
===== Chat, Messaging, and Live chat =====
 +
The following NLP issues relate to chat, messaging, and live chat:
 
* Small talk is not detecting  
 
* Small talk is not detecting  
 
* Topic not detected for an incoming chat request
 
* Topic not detected for an incoming chat request
Line 22: Line 24:
  
 
== Step-by-step debugging process ==
 
== Step-by-step debugging process ==
The debugging process steps include the following:  
+
The debugging process includes the following steps:  
  
 
[[File:NLP debug process flowchart.png|frameless|470x470px]]
 
[[File:NLP debug process flowchart.png|frameless|470x470px]]
Line 29: Line 31:
 
[[File:Channel test.png|frameless|756x756px]]  
 
[[File:Channel test.png|frameless|756x756px]]  
  
*'''Test the channel''': Every channel Email or chat has a test button. Most listener or channel-related issues are isolated with this test. Check whether the right topics and entities are getting detected along with the confidence scores.
+
*'''Test the channel''': Every Email or Intelligent Virtual Assistant (IVA) channel includes a test button in the user interface. You can isolate most listener or channel-related issues by clicking this button. Check whether the system detects the right topics and entities together with the confidence scores.
  
  
 
[[File:Test text analyzer.png|frameless|671x671px]]
 
[[File:Test text analyzer.png|frameless|671x671px]]
*'''Test the text analyzer''': Open '''''Channel > Text Analyzer > iNLP > Settings > "Open text analyzer rule"'''''. If the error shows up in this test interface, then the error is in the channel interface.  
+
*'''Test the text analyzer''': Click '''''Channel > Text Analyzer > iNLP > Settings > Open text analyzer rule'''''. If the error displays in this test interface, then the error is in the channel interface.
 
[[File:Text analyzer run.png|frameless|697x697px]]  
 
[[File:Text analyzer run.png|frameless|697x697px]]  
  
* '''Test the topic and entity models''': The models are essentially decision data rules. Individually, the models can be tested by launching the models from the text analyzer. <br />[[File:Model testing.png|frameless|429x429px]]
+
* '''Test the topic and entity models''': The models are essentially decision data rules. Individually, you can test the models by launching the models from the text analyzer. <br />[[File:Model testing.png|frameless|429x429px]]
**'''Test the model settings''': Models operate at sentence level and document levels. In the test interface, check the results as per the model usage in production. Accordingly compare the confidence score results.
+
**'''Test the model settings''': Models operate at both sentence and document level. In the test interface, check the results as per the model usage in production. Compare accordingly the confidence score results.
**'''Test the confidence scores''': Model confidence score cutoffs(typically >70%) are used as a filter to remove low score topics and entities.
+
**'''Test the confidence scores''': The system uses model confidence score cutoffs (typically more than seventy percent), as a filter to remove low score topics and entities.
 
== Other known issues ==
 
== Other known issues ==
* '''The language is not detected''': This usually happens when the input text is short and may not be a complete example. The expectation is that the input text has at least more than 4 words in the sentence.
+
* '''The language is not detected''': This issue typically takes place when the input text is too short and might not be a complete example. The system expectation is that the input text consists of at least more than 4 words in a sentence.
* '''Incorrect language is detected''': If there are a combination of languages in the input text, then the resulting language may not be correctly detected. In the text analyzer advanced settings, you can configure the NLP engine to fall back on a language when no language is created.
+
* '''Incorrect language is detected''': If you use a combination of languages in the input text, then the system might not correctly detect the language. In the advanced settings for a text analyzer, you can configure the NLP engine to fall back to a specific language when no language is created.
* '''Training lost after instance restart''': The text analytics repository may be pointing to a temporary directory and can result in data loss. Update the repository in Prediction Studio.
+
* '''Training lost after instance restart''': The text analytics repository might be pointing to a temporary directory and can result in data loss. Ensure that you update the repository in Prediction Studio.

Latest revision as of 12:06, 23 December 2021

Troubleshooting NLP

Description Troubleshooting NLP issues - so that customers can self service NLP issues before reaching GCS.
Version as of 8.2
Application Pega Platform
Capability/Industry Area Conversational Channels



How to troubleshoot NLP issues with email, chat, and messaging[edit]

Text analytics and natural language processing (NLP) issues are related to either text categorization (topic, small talk, sentiment, and language) or text extraction (entities, keywords, and auto tags). You can handle most of the issues in the channel interface by using Dev Studio. However, advanced issues might also require access to Prediction Studio. Information presented below covers the majority of NLP issues that you can address by just using the channel interface in Dev Studio.

Symptoms of a potential NLP issue[edit]

Email[edit]

The following NLP issues relate to email:

  • Topic is not detected
  • Topic is detected but the case is not assigned
  • Multiple mentions of the topic are not detected or I am not able to restrict the action to the first occurrence of the topic
  • Language is not detected
  • Model not detecting topic
  • Case properties are not detected
  • Feedback data not reflecting
  • Signature and greetings in email body part of entities detected
Chat, Messaging, and Live chat[edit]

The following NLP issues relate to chat, messaging, and live chat:

  • Small talk is not detecting
  • Topic not detected for an incoming chat request
  • Escalation to agent not getting detected

Step-by-step debugging process[edit]

The debugging process includes the following steps:

NLP debug process flowchart.png


Channel test.png

  • Test the channel: Every Email or Intelligent Virtual Assistant (IVA) channel includes a test button in the user interface. You can isolate most listener or channel-related issues by clicking this button. Check whether the system detects the right topics and entities together with the confidence scores.


Test text analyzer.png

  • Test the text analyzer: Click Channel > Text Analyzer > iNLP > Settings > Open text analyzer rule. If the error displays in this test interface, then the error is in the channel interface.

Text analyzer run.png

  • Test the topic and entity models: The models are essentially decision data rules. Individually, you can test the models by launching the models from the text analyzer.
    Model testing.png
    • Test the model settings: Models operate at both sentence and document level. In the test interface, check the results as per the model usage in production. Compare accordingly the confidence score results.
    • Test the confidence scores: The system uses model confidence score cutoffs (typically more than seventy percent), as a filter to remove low score topics and entities.

Other known issues[edit]

  • The language is not detected: This issue typically takes place when the input text is too short and might not be a complete example. The system expectation is that the input text consists of at least more than 4 words in a sentence.
  • Incorrect language is detected: If you use a combination of languages in the input text, then the system might not correctly detect the language. In the advanced settings for a text analyzer, you can configure the NLP engine to fall back to a specific language when no language is created.
  • Training lost after instance restart: The text analytics repository might be pointing to a temporary directory and can result in data loss. Ensure that you update the repository in Prediction Studio.