Software-as-a-Service business with all the virtues that it purports also demands that the service provider be agile. Agile, not only, in terms of the way product is built but delivered and managed. Unlike in the traditional software days, subscription revenue model requires that SaaS solution provider measure every process and continuously adapt based on the findings. The primary goal behind it is to identify opportunities to drive down the cost of acquisition and cost of service delivery.  The insights gained therein also feed client services, marketing and product teams with up-sell opportunities, campaign inputs and future roadmap items for value added services.

I have been working with a SaaS provider to help them define metrics that should be measured, monitored and correlated to business metrics. Thought it would be useful for others who might be in SaaS business. There is no denying the fact that the following critical metrics

• Annual Contract Value (ACV),
• Total Contract Value (TCV),
• Monthly Recurring Revenue (MRR),
• Average Monthly Revenue per Customer (AMR)
• Churn Rate
• Customer Acquisition Cost (CAC)

should be tracked and measured from a finance/profitability point of view, but in this post, I will focus on operational metrics that help you build customer success.

Usage Metrics

• Application Logins: it is always a critical statistic to measure how many users are signing onto the application. It not only talks to the scale of the application, it also demonstrates the critical nature of the application. If you broke down the logins by the role of users in the application – for example a HR Manager versus a VP of Talent Management, and individual usage patterns should help drive focus areas in your product roadmap. Marketing can use this same information to create day-in-a-life documents or case studies on the critical nature of the application. Operations can use the metric # of logins as a way to demonstrate the scale, SLA and also determine peak/off-peak usage patterns for performance benchmarking, planning scheduled downtime etc.

• Time Spent in the application: Time spent in the application is something a company should always be proud of. It talks to the stickiness of the application. While that is something marketing can use to demonstrate the value of the product, product teams should inspect the same metric to identify potential areas for optimization. In this day and age, users seek smart business processes that are not click-hungry and easy to accomplish. So users spending longer time on a given process could imply productivity loss and in the long run could lead to unhappy users. Client Services should look at this metric to identify opportunities for training (or lack thereof). It is always critical to benchmark a typical lifecycle of a business process and see if there are reasons to be concerned if deviation from the benchmark is large.

• Access Mechanisms: As such most SaaS applications are accessed using a browser. But with the browser wars looming again it is important for a product team to measure and compare the different browser usage – Firefox, Chrome, Safari and the Goliath – Internet Explorer. Besides the different browsers, it is critical to measure the versions of each. With effective logging of sessions, this information should be easy to capture. Besides browser, it is also important to capture information around
  • Desktop Operating System (Windows, Mac, Linux)
  • Monitor Resolution
  • Platform (Desktop Vs Mobile)
  • Languages used
  • Bandwidth used (DSL, T1, Dial-up)

  Product teams can use this information to improve testing coverage,   support for specific browsers (versions). Marketing can use this same information to highlight to the broad capabilities of the product/platform. Sales will require this information for the RFP they complete as part of a sales deal.

• Source of Users: Where your users are accessing your application from is a key important metric from multiple angles. If you are in Sales, you probably know your customers in various geographies. Facts about the density of users from a particular geography might indicate better adoption rates and need for increased sales efforts. For product team, this might drive decisions around caching strategies, internationalization or localization needs, increase in latency based tests. If you are in marketing you will now be able to leverage customers from various geographies to provide you localized references in marketing efforts.  Operations can factor this information to gauge the coverage of the redundant data centers created to cater to global users.  For those with one active data center, this could provide insights to support that second data center plans you were putting in place.

• Business Transaction Density (Quotes created, Search conducted): Capturing metrics around the key activities performed in the application such as creating orders, creating customers, search conducted, user roles created, projects created, surveys conducted, documents uploaded are all great ways to measure the coverage of usage of the application. The product team can use this information to crosscheck with the roadmap and identify the cause for lack of usage in certain areas. Follow that up with discussions with customers that requested those features to better understand the effectiveness of the product feature. For instance a large number of document uploads might indicate companies using documents in lieu of  structured business process or compliance to a governance mandate. Knowledge about such processes might lead to identifying opportunities for expansion of product footprint. Operations can use the same metric of large number of document uploads to determine if storage configuration should be optimized or potential for using de-duplication technology.

Operational Metrics provide additional insights in the user behavior and indicate hidden opportunities to improve.

• Tickets logged: Not the most favorite metrics for any constituents in the service provider company.  But I have a bright side. It is much better than not have any tickets at all – atleast you know your product is used. While it is standard to bucket tickets into product areas, I recommend you break down tickets into those logged by new users, critical areas of business process and specifically those logged during critical dates (month end, quarter end and year end). The easier you make new users to adopt the application, reduce the instances of fall-over the more purpose with which they will use the application. The more they will talk about your application. Conversely, the more troubling it is to get accustomed to the application the sooner they will desert it. Critical areas in business process and critical dates need no highlighting as to why they are important.
• Timeouts: Timeouts in my opinion are the worst kind of issues. In addition to creating a bad perception of the product, they also could point to infrastructure issues, missed test cases. They also put client services in a bad spot where they cannot explain the cause unlike a product deficiency. Considering that it is not always possible to root out all time out issues due to the varying nature of access (cable, dsl etc), it a great idea for client services to have a “Have you checked this?” list. Worst of all are those that happen during crucial demos to prospects.
• Downtime: While unplanned downtime is bad and puts your CEO in the news for the wrong reasons, planned downtime is equally painful from the customer point of view. Given the round-the-clock nature of world we live in, people extend their work lives to evenings and weekends. So having excessive downtime and more so, those that went over the announced window need to analyzed. No one likes to work on weekend and if you cancel plans to work on weekends only to find out that downtime window has been extended would not make for a happy user on Monday.
• User growth over time: This is a no-brainer of a metric. Tracking user base growth and charting the patterns gives you a view of buying habits, correlation of success of campaigns and what you do well. It also show the times when you should increase thrust on your sales/marketing campaigns.

This was a representative sample of long list of metrics we identified and started tracking. The list extended to sales, marketing and implementation to track success in converting leads, success in converting trial users, investment done in implementation cycles, training cycles etc – I am sure you catch the drift. If you have channels, then you will need another set of metrics to measure the effectiveness of channels and give you insights into what does (not) work.

All these metrics once identified and tracked, can be part of a dashboard that all employees in the company has access to. Make sure to include those as discussion items in company meetings and goal settings for each executive.

In an on-demand business, the risk is heavily tilted towards the software vendor. It is incumbent on the company to automate, measure, monitor key statistics and adapt the business based on those insights. Any/All decisions made in roadmap, client services, market can critically impact the success/failure of the company.

About Subraya Mallya

Subraya Mallya is a technology executive, entrepreneur involved in conceiving, building software businesses. In this blog, he focusses on business and technology strategies around Business Model, Product Development and Marketing. Connect with me on Twitter - @subrayamallya