Compare Algorithm Outputs to Reveal Underlying DataSet Structure

Problem: Data has noise. Data sets must be cleansed, normalized and curated. But you never really can tell for sure which algorithms to apply unless you compare results.


  1. Plan to generate different visualizations  of your data set,
  2. Plan to create and various transforms on your data set
  3. Run through a number of typical algorithms on each view of your dataset.

Purpose:  Clarify which data transformations will reveal the structure of the dataset that underlies your problem.


OCR text classification

One of the main decisions we have to make upfront is once we apply OCR to incoming unstructured data how do we classify it according to ontologies in that specific business, domain or even industry.

Domain ontologies serve as metadata or mid ontologies where we can consolidate unstructured and structured data that may have very little overlapping or mapping ontological schemas but can be used to consolidate and join them for subsequent congregation aggregation and data lake consolidation efforts.


Pattern one:Routed ingestion

In this pattern we ingest content whether from an email or transcription of voice to text and then we classify then we perform tone and sentiment analysis will use natural language understanding to accomplish the above three then we apply a set of heuristics possible if then rules or machine learning based classification rules and dialogs in an intense based architecture and initiate a workflow which may be a routing to the appropriate Houman in the loop


Sub patterns and variations of data in Machine intelligence

Identifying patterns is Essentially the identification of recurring patterns of data

We attach significance to a specific pattern. Something recurring may or may not be significant. However once we attach intent to the patterns that we are interested in or we provide heuristics for possible sub patterns which are very Asians on the overall pattern then the intent beast architecture starts becoming a template for parent identification and can be utilized in machine intelligence.