Explore Data Through Written Word

Adapting to change in financial services requires efficient ways to understand and communicate actionable insight from data.  Data visualization is the standard by which to uncover insights, trends, and hidden opportunities.  However, we now have access to very practical tools that allow us to explore data through written word, or Natural Language Generation (NLG).  When it comes to telling a story from your data, that story could be automatically written for you.

Collect, Prepare, Analyze and Visualize Data

First, a quick word about data exploration.  In the end, the goal is to discover and communicate actionable insight from disparate sources of information.

  1. Data collection is the process of identifying the right data sources and integrating them in a meaningful, accessible, and useful way.

  2. Data preparation tools (in tandem with collection) help to clean and identify relevant relationships within the collection.  Some tools can even advise you to not waste time on a source that has little chance of being meaningful.

  3. Data analytics tools analyze and (should) provide useful historic, real time, or predictive results.  Outcomes are better if the preparation tools did their job and you’ve collected relevant data.

  4. Data visualization helps to communicate analysis through a variety of simple or sophisticated graphic models and can allow you to interactively dive into the data.

Visualization Has a Design Dependency

A problem with data visualization is that the type of graphics used can either be too simple or too complex.  A bar chart is clear and can describe volumes and trends over time but the significance of the trends are left to a viewer's interpretation, rightly or wrongly.  More sophisticated graphic models like heat maps, scatter, and and butterfly charts express multi-dimensional information but can be confusing to the untrained eye.

Visualization is effective but it has a design dependency.  Unless a dashboard is designed very well, the story in the data can get lost.  It takes clear intent and expertise to visually communicate well, often with great success, but it is real work.

Data Exploration Through Writing

Writing is work too.  It is easier to write a lot of detail than to write clearly and concisely.  The ability to summarize concepts from detailed data is a real skill, both for people and machines.  It takes time to compose a good email.

To point 4 above, we can further optimize data analysis and communication via tools that automatically generate written narratives from data.

Below is an example of how the same data that generates a chart can generate concise, well written text.  The automation not only provides details that the graph doesn’t easily show but the output is well formed for easy communication (i.e.: copy and paste into an email).

This example is the analysis of trading activity using Excel that has an NLG plug-in.  The role of the chart is minimal.  It is used only to model the data at a high level and trigger the NLG plug-in.  

  • Bar Chart Showing Trading Activity
  • Automated Text:  “We can see that a few trading desks account for more than 80% of the total trades. There is obviously a dominant group of trading desks. This group is composed of six trading desks:...”(detailed list and percentages follow).

  • Action:  Filter chart to show only cancelled trades.

  • Automated Text:  “What stands out in this situation is that a few trading desks account for more than 50 % of the total. This group is composed of three trading desks:....”(detailed list and percentages follow).

Evaluation Criteria - Conciseness and Integration

When picking a NLG solution for your organization, it is important to evaluate how concise the written output is.  As noted above, it is easier to generate details than to summarize clearly.  Secondly, it is important to evaluate how easily it integrates with standard business intelligence tools.  The primary benefit of visualization is the ability to interactively drill down into detail.  Narratives need to be instantly updated and in synch with that exploration.

Summary

When it comes to telling a story from your data, practical tools now exist in which at least a portion of the story can be automatically written. This is one part of a larger solution set by which to discover and communicate actionable insight from disparate sources of information.