As more organizations are realizing the value in data, they strive to get better at using it. Tools are improving and innovation is happening. Much of the innovation we see today is happening in departmental silos, where data scientists are diving deep into technical analysis driven by the department leader with a specific vision.
We see a need for innovation at the top. It can serve to initiate the framework of a new data program or as the capstone to a program already in place.
The Panoptive Mission
Establish the optimum route from business question to informed answer.
To achieve this, we can’t just build another tool. We’ve developed a solution with the user at the center. This can be anyone with a business question. While it’s typically someone in leadership who initiates the question, someone in their hierarchy is often responsible for the answer and could be in any role or level. In this article, we’ll refer to this person simply as the “user,” since the function of answering questions can be performed by anyone.
In the real world, questions take the form, “why are sales down this month?”. For the user to answer the question thoroughly, they take several steps:
- Confirm the numbers’ accuracy
- Break down the number to see if there is a trend in a segment (Is it a given product? A certain region? A customer segment?)
- Check for seasonality (Does this drop happen every year?)
- Look at the timeline (Is this a longer term trend or truly isolated to the month? Other patterns?)
- Investigate changes to influencers of the number (Ad spending changes, distribution disruption, IT downtime impacts, competitive pressures, increase in negative social media sentiment, etc.)
What seems a fairly innocuous question can burn up the next week. Panoptive dramatically simplifies this, through several approaches.
[Tech Note: Both front-end and back-end approaches are used: data gathering, data quality, governance, definition, contextualization, charting, decision support, performance management, anomaly detection, and alerting.]
We’ve developed ways to automate, streamline, or better enable what users are doing today. A recent study shows data analysts spend more than half their time accessing, blending, cleansing, or otherwise preparing the data.
Our product is solving these problems with automation.
However, we can’t solve those problems with another reporting tool, BI tool, ETL tool, data prep tool, discovery tool, or data warehouse. These tools have their place. But for the user trying to answer a question, it’s one more layer of technology in the way. Another piece of technology they’ve had to master. Another piece of technology the organization has had to purchase, install, and support.
[Tech Note: In the case of BI tools, the metadata layer that makes it easy to use (the universe, repository, …) does hide complexity, but it also hides the understanding of how exactly the data is being manipulated. You can give someone a number, but you can no longer answer a follow-up question.]
How Panoptive is Different
We focus on the business questions. We put the executive and systems user at the center. We do it better than traditional data tools by creating a solution that addresses the friction in the process.
What Do Users Need?
1. Context
The issue: Users can’t easily answer the “why.”
Standard reporting supplies what happened, where it happened, when it happened. The “why” is elusive. To some degree, this investigative work should be part of the user’s job. But once the answer is discovered it is usually communicated once, and that knowledge dies there.
Example
Sales for a given product is down one week. The user is asked the question why sales are down. After extensive investigation, the user discovers a primary competitor lowered prices for that product. It answers the question and ends up on a PowerPoint slide somewhere. Then what? Will it be remembered? Is it shared with other teams who may end up doing that same research? Will this issue affect the monthly numbers, and be asked again? What about next year, when this week gets compared to a future week?
Panoptive Solution
- We allow tracking of these events.
- We tie these events to any metrics that may be impacted.
- We display this context by default when you look up an affected metric.
2. More access to more data points
The issue: Users don’t have access to the data they need. This isn’t a question of user permissions. It’s intentional, and there are good reasons for restricting access for users. But there are also good business reasons for establishing controlled methods to extract data.
The problem of access is the top issue sited by this study.
None of the tools mentioned above (BI, reporting, etc.) solve the problem of access. Building new data sets into a data warehouse could solve this issue, but the cost and development effort tends to prevent this from happening quickly (estimation, approvals, data modeling, ETL design and development, BI design and development).
Panoptive Solution
We’ve perfected the process of quickly pulling new data sets into our data model, and making them useable. We don’t require the typical 3-6 month project, taking resources from the IT staff. We’ve eliminated the need for report development, estimation, data modeling, ETL design and development, and BI design and development. [Tech Note: This is not just a Hadoop trick where we dump the data into an unstructured zone and have to figure it out later. It’s structured and useable.]
We only require a little knowledge about the data set. [Tech Note: Think of a “data set” as a table of data… csv, xml, or the results of any database SQL statement.]
After initial setup and connection to a given data source, adding incremental data sets can take just minutes and can be self-service.
What do we do with these data sets?
Once we ingest a data set, we build every possible combination of those data points as a series of metrics… the beginning of your comprehensive metric catalog.
[Tech Note: This is similar to what a cube would provide, but we leave in a relational database model for ease of access.]
Example
Remember our sales example from above? A Panoptive solution would put the answer to those questions at your fingertips. Every breakdown and segment of the data. Every level of time. Every metric influencer.
3. Consistency
The issue: Users across the company don’t have consistent data.
Few things are more unsatisfying than reconciling the difference between two reported numbers. It happens regularly. Two managers come to a meeting showing different results for the same metric. It’s always explainable (different sources used, different criteria, pulled at a different time), but the exercise is still largely a fool’s errand. It takes time, delays decisions, and rarely changes the directionality of relevant trends.
Example
User #1 pulls a sales report. Their manager prefers it when they exclude the new subsidiary. They also know of some sales activity logged as part of testing, so they exclude that too.
User #2 works for another department. They’re not aware of the testing activity, and they didn’t know the new subsidiary was part of the data.
It’s easy to see how the numbers can get out of sync.
Panoptive Solution
We establish a metric catalog as the single source of truth. Each report that user #1 and #2 pull is explicitly labeled with their definition. The catalog is shared with other departments. Metrics are explicitly defined and definition also shared. When changes need to be made they’re made in one place.
Q: Why is Panoptive’s metric catalog different than a data warehouse?
A: A metric catalog can exist for companies without a data warehouse. For companies with an existing data warehouse, a metrics catalog can “sit atop” the warehouse, further simplifying and standardizing the data.
Q: Why will this work as a single source of truth? We’ve heard that before.
A: Three things… speed, comprehensiveness, & ease of use. Once the catalog is established, if it’s easy to access, easy to make changes, and the most comprehensive source of metrics, it will achieve this.
Scratching the Surface
With these top three user needs, we’re just scratching the surface. We’ve assembled a list of 40 pain points that users experience on a regular basis, and we’re still gathering more.