Risk Management at Scale Part 3: understanding what moves the needle

Risk Management 3.JPG

{If you missed the first two parts of Risk Management at Scale, read those first and then come back here}.

If you’ve made it this far, kudos on your drive towards reducing risk. Setting up quantitative and qualitative measurements (Part 1), and the process of collecting data (Part 2) are the prerequisites for the real work: analyzing the data. Well, there’s one last step between collecting and analyzing… and that’s physically getting the data.

Many early-stage companies have a limited suite of tools for data export, collection and analysis. You may also receive data in a basic format, like a spreadsheet, SQL export, or a CSV from your internal database. An engineer or data scientist may be able to give you a massively sliced export of the entire database, but it’s often beneficial to keep things simple. Use the software tools you’re comfortable with for this first pass, and keep in mind what additional data you would want next time.

Okay, data is exported and you’re ready to go. But wait, are you the right person to start digging into the analysis? Do you know how to read between the lines? Do you have a clear performance indicator to keep in mind while you are researching? It’s okay if you don’t know right now, but it’s not worth your time if you start exploring without a destination in mind. Take a moment to understand your larger business problems to find those KPIs. Imagine you found a treasure map but there’s no big X indicating where it’s buried. Not much help, is it?

Talk with your team and figure out the problems you’re trying to solve. You can ask everyone what they think is important and use that as a guide… OR go in uninfluenced and see for yourself what is important. Both options have their pros/cons so proceed with what you think makes sense. (Note: If you choose to go in with a hypothesis, a great place to pre-investigate would be your post-mortems. See the post-script for more information).

Let’s use a basic scenario: you have a CSV, are using Excel, and have some moderate skills with filtering / pivoting / vlookups. As a personal preference, we recommend keeping your raw data untouched and pristine in its own worksheet, and then copy-paste everything to a new tab to do your filtering. This ensures you can always return to the full set when you want to take a different approach.

Sort, filter, sort, filter. As you start to poke around, you’ll end up either finding a) the needle in the haystack → outliers, or b) a haystack → trends. Trends are the 80% you should focus on, tackling the bulk of your customers first. For the most part, a single outlier doesn’t tell you anything useful for creating new frameworks to prevent risk. It would be hard to build a system to prevent churn for the one customer that decided to pivot from selling iced tea to adopting blockchain technology (true story).

Look for the haystack, or rather the multiple small haystacks that can be indicators. (If we use Google Search as an example, the big haystack is ad revenue, and the small haystacks are time on site, active users, searches per day, etc). And make sure to document your filters / sorting, so you know what your dataset contains. If someone else with the same data set did the same analysis with the same filters, they should arrive at the same conclusion.

Not all of your discovered trends will be groundbreaking, but document them anyway. Each trend can be an individual clue to unlock a bigger theme within the data. And while you may find a few outliers that match up, don’t get distracted trying to build a case of outliers. Remember: focus on the 80%.

How do you know what’s important? A good starting off point is assume that everything is important! The discovery of “Our customers like it when we make them money” is obvious, so dig deeper. Do they want a 5x ROI, where they pay you $1 a month, and want to make at least $5? Is it more or less? Are there qualitative deliverables that don’t have direct revenue attached to it but can keep a 1.1x customer around for years? Finally, strive for finding trends and causes, not coincidences. It’s critical to understand whether a data point actually drives retention and customer happiness or if the impact is created by another factor.

Customer example: After a thorough analysis, we drew a line on what an acceptable ROI multiplier was for all client tiers. Then, we noticed a second data point: uneven contract vs service levels. Customers would sign contracts for a certain number of widgets per month, and even if they were able to exceed their monthly revenue goals with 10% fewer widgets, they still felt ‘cheated’ out of their full order. This feeling was mitigated going forward by having CS assessing and clearly documenting contract obligations and promises. It also became a helpful data point to cross-examine customer risk. From this, we were able to build the right framework (see Part 4!).

Finally, get a second and third set of eyes on your findings. For person #2, it’s best to have someone who knows the customers, the space, and are preferably close to your team. They are there to help you see the forest, as you’ve spent so much time in the trees. For person #3, go to another department. Their viewpoint is helpful as they can approach it from the product, marketing, or sales mindset. They may also have knowledge you weren’t privy to (i.e. an email blast went out on a certain day and that caused a huge lift in traffic and server costs) that can help color your findings in a new light.

Analysis is complete. Now, we’ll learn how to leverage your data to prevent risk (Part 4) and do it consistently at scale (Part 5). To get updates when we publish the additional parts of this series, be sure to follow Sandpoint Consulting on LinkedIn.

For more information about Risk Management, or to request a customized Risk Management Workshop for your team, send us a note at contact@sandpoint.io.



post·mor·tem  / pōs(t)-ˈmȯr-təm / noun: a process, usually performed at the conclusion of a project, to determine and analyze elements of the project that were successful or unsuccessful

If your relationship with a customer has concluded, chances are it was unsuccessful, since they decided to stop using your product. Your team should be collecting post-mortems for every churned customer. More information can be found in this blog post.

Risk Management at Scale Part 2: collecting the data

Risk Management 2.JPG

{If you missed Risk Management at Scale Part 1, read it first and then come back here}.

“I wonder how many people live here.” - US founders, 1789 → US Census
“Do you know how fast you were going?” - Highway police, present day → Radar gun
“Are customers using the product the way we thought they would?” - You → ???

For better or worse, humans are obsessed with measurements and metrics. It helps us to compare our progress over a time range with our peers or our competitors.

To start, let’s define an important term that should help turn those above questions marks on product use into actions:

te·lem·e·try / tə-ˈle-mə-trē / noun: the science and technology of automatic measurement of data from remote or inaccessible points and transmitted to receiving equipment for monitoring.

Your team may already be accomplishing this telemetry: user analytics from Mixpanel to measure traffic, Zendesk reporting to keep track of tickets created per customer, and even Google news alerts for important customer events like acquisitions or board member changes. All of this is great information for your customer success team to collect and have top of mind for their next call. This is quantitative data.

But speaking of the call, how do you measure your relationship? This is qualitative. Below are examples of phrases to listen for that can help a CSM understand their customers temperature.

  • Are they talking about events far into the future (i.e. beyond their upcoming renewal date)?
  • Has your day-to-day mentioned that their manager is leaving and a new person is taking over the department?
  • Have they cancelled the last couple calls with no explanation?

While there is no specific KPI connected to these remarks, it tells you something about their specific experience interacting with the product and CS team. Typically, we recommend customers use either school (A - F) or traffic light (green - yellow - red) grading to evaluate the relationship. Updating the score, even if it’s refreshing the same grade because the relationship is still great, helps to ensure that everyone in the organization knows this is accurate.

Where you store the relationship score really depends on your suite of tools and budget. There are customer relationship management tools, like Salesforce and Gainsight, that have this functionality built-in. Changes in score can also trigger specific playbooks based on a positive or negative movement. In addition, we’ve seen companies simply use a shared spreadsheet with only four columns: customer, assigned CSM, score, and date updated. It’s worth noting that this seemingly simple spreadsheet, when organized correctly, can become the foundational documentation for when you upgrade to a CRM tool, as it provides a clear template for your implementation.

Working in tandem with CSM scoring, your product and engineering teams should have proprietary systems and metrics built into the software you sell, which helps understand customer behavior. When is the last time they logged in? Is the day-to-day only using one feature? Is the customer VP clicking on the ‘Review Plans’ page, comparing their basic package to the premium version?

Collecting isn’t the hard part (hopefully). Once you have all this data, ask yourself if it’s accurate. And, if it’s not accurate, why? It’s beneficial to absolutely no one if you find reasons to trim numbers: “Oh, ignore these customers because they are too big / small / odd.” While this practice may result in the pretty up-and-to-the-right chart to show your board, you didn’t learn anything and don’t know where to focus your efforts.

It’s also important to build a culture of transparency within your customer team.  If your team members feel overly incentivized to fib on their customer’s happiness (giving green when it should be yellow or red), you’re up a creek with no paddle.  It is much better to know there’s an issue and have a plan for action than to be falsely confident and blindsided by bad news. A customer should rarely, or never, go directly from green to churn.

Let the story unfold from the full data set, and accept it as the current picture. Ignorance is not bliss. It’s better to know that the numbers are not great, rather than have “good” heavily edited data… and then be surprised by a spike in churn.


So, you’re collecting the data. Next we’ll get our hands dirty figuring out what all this data tells us about today, and how that changes tomorrow (in Part 3). From here, we’ll learn how to leverage your data to prevent risk (Part 4) and do it consistently at scale (Part 5).  To get updates when we publish the additional parts of this series, be sure to follow Sandpoint Consulting on LinkedIn.

For more information about Risk Management, or to request a customized Risk Management Workshop for your team, send us a note at contact@sandpoint.io.

Risk Management at Scale Part 1: quantitative vs. qualitative


No matter what you are selling to another person or company, whether it's a service, software or a cup of coffee, understanding risk factors and designing easy, scalable ways to stay ahead of churn are paramount to your company's success.  In order to begin this process, you must start thinking about your customers from their perspective and find ways to determine their "health" as it pertains to their relationship with you.

Most of our clients have some semblance of a qualitative health score on their customers when we begin working with them.  This usually comes in the form of either a traffic light (green - yellow - red) or letter grade (A-F) score given by a Customer Success Manager (CSM) to a customer based on the perceived relationship.  Sometimes this information is stored in a Customer Relationship Management (CRM) tool (like Gainsight or Salesforce), an internal customer wiki (in Confluence or Asana) or even in a spreadsheet (Excel or Google Sheets).  This data point may be updated regularly (daily or weekly), but more often is only updated monthly, quarterly or even "periodically", which translates to: when I remember to do it or if I'm told I have to...

We'll talk more about the structure of health scores in Part 4: building a framework, and we'll discuss update frequency in Part 5: change management and automation.  In this post, we'll describe the two major ways to evaluate customer health: quantitative and qualitative.  

Let's start with the easy one, or at least the one that seems easy on the surface: qualitative health.

qual·i·ta·tive  /ˈkwäləˌtādiv/ adjective: relating to, measuring, or measured by the quality of something rather than its quantity.  "a qualitative change in the undergraduate curriculum"

To put it simply, this score "bucket" relates to the quality of the customer's experience with your product or service.  Are they happy with it? Are they delighted by the experience?  How do they feel? 

Every interaction with your customer can give you a clue about their happiness with your company, your product or service and your team.  Accurately capturing this information after each interaction is vital to ensure that you are mitigating risk as it arises.  The tricky part is not capturing this information, per se, but doing so accurately.

The place that a lot of companies and team members slip up is through the assumption that "no news is good news".  Regarding customer health, this is ABSOLUTELY NOT the case.  No news is usually terrible, horrible, very bad news. 

In general, the less engaged your customer is with your company, the higher the risk of them leaving.  Why?  Because it's really easy to "switch vendors", but it's really hard to fire a person.  When your customer is engaged with your company through your customer team, they are interacting with people.  This interaction creates relationships and, done well, makes your customer very hesitant to "fire" you.  Without that relationship, you're just another tool that can be replaced with a newer, cooler, better, more interesting tool.

We'll dig deeper into this subject in Part 2: collecting the data.

Now let's get into the real meat and potatoes of customer health: quantitative scoring.

quan·ti·ta·tive /ˈkwän(t)əˌtādiv/ adjective:  relating to, measuring, or measured by the quantity of something rather than its quality."quantitative analysis"

This is where a lot of our customers freeze up.  How do you quantify your customer's health?  Isn't health something soft and "squishy", like relationships and happiness?  The answer is... well, sorta. As discussed above, your customer's happiness is a key part of their overall health, but there are tangible, measurable aspects of their interaction with your company and your product that directly impact their happiness.  How much value are they seeing out of your product or service?  How often do they use your product or engage with your service?  How much impact is your product or service creating in their business?

The great news is: all of these items are quantifiable, collectible and potentially meaningful indicators of the health of your customers.  Throughout this series, we'll describe not only how to collect the data and analyze it (Part 2 & Part 3), but how to leverage it to prevent risk (Part 4) and do it consistently at scale (Part 5).  To get updates when we publish the additional parts of this series, be sure to follow Sandpoint Consulting on LinkedIn.

For more information about Risk Management, or to request a customized Risk Management Workshop for your team, send us a note at contact@sandpoint.io

Welcome to the Sandpoint Consulting Customer Success Blog


Well, hello!  

Welcome to Sandpoint Consulting, where we are driven by customer success.  We work directly with teams at early-stage startups to ensure their customer base stays healthy and continues to grow.  Our focus is on B2B (business to business) software, SaaS (Software as a Service) and Enterprise software companies.  Here is our mission statement: 

Empower growing and soon-to-be-growing companies to retain customers effortlessly while improving team efficiency and happiness.

This is our blog where you will find tips, tricks, musings, learnings and opinions from our Co-Founders, Emily Speer Ryan (that's me!) and Adam Croce.  We will center primarily on customer success-related topics, including structuring your business for customer success, customer delight through data, communication for success and much more.  

Customer Success is a Business Imperative

The world is full of amazing, innovative and valuable technology; just open up your smart phone and page through the dozens of apps you've downloaded!  In this fast-paced and novelty-driven age of tech, how do you ensure that your software stands the test of time and keeps its customers coming back?  Lead with Customer Success.

But what is Customer Success with a capital "C-S"?

You are probably familiar with the phrase Client/Customer Service and you are certainly familiar with Client/Customer Support, but have you heard of Client/Customer Success?  It's a fairly new phrase and is primarily used in the technology industry.  When I think of Customer Service and Customer Support, I think of calling a help desk to answer the question I have or solve the problem I face.  I imagine a room full of friendly, helpful people who wait for me to call so that they can respond to my need.  They're reacting to my need.  They are re-active.

Customer Success is different.  Customer Success is pro-active.  Customer Success directly collaborates with your enterprise customers and forms strategically aligned relationships with your customer champions and stakeholders, ensuring value delivery, anticipating needs, promoting growth and expansion and, most important, retaining their business.

But isn't that what Account Managers do?

Yes!  Customer Success is the evolution of the Account Manager function.  Customer Success builds the relationship while focusing on driving value from your software into the customer's business.  Many business models saw something similar to this:

Sales Director --> Account Manager --> Customer Support

(new business) --> (expansion & renewal) --> (troubleshooting & help desk)

In this model, the customer's experience is either sales-oriented or reactive.  Every person at your company that interacts with the customer is either trying to actively sell them something (or will sell them something at some point) or they are reacting to a negative experience the customer is having (Help me do this.  Is this broken?  How do I...?)

The Customer Success model looks something like this:

Sales Director --> Customer Success Manager --> Customer Support

(new business & upsell) --> (relationship & value assurance) --> (troubleshooting & help desk)

Now the customer, especially your most valuable enterprise customer, has a strategic partner in your business.  They have a trusted advisor who isn't trying to sell them something.  They have a person to get excited about strategy and business outcomes with.  They have someone focused on their success.

That sounds like a lot of work with very little upside...

It costs five times as much to attract a new customer than to retain an existing one. (source)

Let that sink in.

Now think about this: increasing your customer retention rates by 5% increases profits by 25% to 95%. (source same)

By investing in Customer Success, you are investing in the future of your company.  We can help you convert those sales into long-term, case study-ready, enthusiastic evangelists, and we can help you do it at scale using data, lightweight process and clear, consistent communication.

Exploring Customer Success at Emerging Software Companies

This Customer Success blog builds on the experience Adam and I have designing simple, inexpensive and "scrappy" solutions at growing and soon-to-be-growing companies.  It shares with you our passion for scalability, repeatability and simple, practical company solutions that focus on substance rather than flash.  

And in an industry where everything changes in the blink of an eye, we focus on speed and agility to ensure you'll see success quickly and be able to flex the process as you grow and scale.

Rules of the Road... er, Blog

What we publish in these blog posts represent our views, opinions, experiences and ideas. Where applicable, we will cite our sources and encourage you to be thoughtful & critical readers.  If you ever have questions or thoughts about a post, we'd love to hear from you: contact@sandpoint.io

You can learn more about me at sandpoint.io/about, on Linkedin or at about.me.  Adam can be found at sandpoint.io/about, or on Linkedin.

Thank you for reading!