July 17, 2009
Gartner SaaS Satisfaction Survey Misleading
Last week, Gartner attempted to derail the Software-as-a-Service (SaaS) movement by issuing a press release regarding its latest customer survey which found that SaaS users are “underwhelmed by their current experience of it and sense that SaaS is not quite the panacea it often promised to be.”
For some reason, it took Gartner more than six months to interpret the results of its survey of users and prospects of SaaS solutions in 333 enterprises in the U.S. and the U.K. which was conducted in December 2008. Maybe because the data didn’t say what they were hoping and it took more time to twist the results into a story that fit their reality and could generate a few headlines.
Gartner’s press release starts by stating that “SaaS is more mainstream and less controversial than ever before”, an important admission from a research firm which a few years ago said SaaS would only represent 25% of software sales by 2011.
But, the research firm goes on in its survey announcement to play the role of contrarian, which is typically held by Forrester, and suggest that all isn’t rosy in the SaaS world.
Gartner’s research vice president Ben Pring says in the release,
“Our research findings did not exactly provide a ringing endorsement of SaaS, in fact I would go as far as to say that satisfaction levels among SaaS users are little more than lukewarm. Although macroeconomic factors would seem to favor SaaS providers, almost two thirds of respondents said that they planned only to maintain their current levels of SaaS in the next two years.”
Although the overall satisfaction level of 4.74 on a 7-point scale among the SaaS users surveyed by Gartner may not have been stellar, the other data revealed in the press release suggests that these ratings are also not as bad as they look. And, other realities of the market and Gartner’s world are also worth noting to bring the survey findings into proper perspective.
Given today’s turbulent economic climate in which many organizations are actually downsizing their operations, maintaining SaaS current levels isn’t necessarily a bad thing. Instead, it demonstrates the staying power of SaaS solutions and how the more flexible pay-as-you-go SaaS subscription approach is well-suited for today’s economic uncertainties.
But, more importantly, Gartner’s press release fails to emphasize the positive news that 90% of the survey respondents are satisfied enough to maintain or expand their use of SaaS solutions, and only 5% are planning to terminate their services, with the remainder considering reducing their subscription levels.
This latter group could be motivated by downsizing in their organizations and would not have had the same flexibility if they chose a traditional on-premise application with an upfront perpetual license fee.
Anyone familar with the economics of the SaaS business knows that 90% renewal rates are not only the norm for the industry but are necessary to maintain a positive cashflow and achieve long-term profitability.
Given that Gartner’s clientele and the primary respondents to its survey are generally IT people who are still learning about the benefits of SaaS and evaluating it from a traditional set of technical criteria, it is not surprising that they might be less satisfied with the SaaS solutions than their business end-users which the bulk of today’s SaaS solutions target.
In fact, there are still plenty of situations in which IT staff are particularly unhappy with SaaS solutions which were acquired unilaterally by individual end-users or business units without IT involvement or approval.
It would be interesting to do a cross-tab analysis of Gartner’s survey results to see if there is any correlation between the low satisfaction ratings and the level of involvement the respondents had in the selection processes.
It also would have been interesting to compare the SaaS satisfaction levels with similar satisfaction ratings among those users of today’s legacy applications. My guess is that their ratings would be well below the SaaS levels, especially given their added costs and inflexibility in today’s tough economic times.
Fortunately, a growing number of IT decision-makers and organizations are recognizing that SaaS is satisfying the needs of their end-users and corporate executives, and are doing their best to help their business users select the right SaaS solutions to meet their operating objectives.
THINKstrategies’ SaaS surveys, in conjunction with Cutter Consortium, uncovered the growing acceptance of SaaS among IT professionals in 2007.
These enlighted IT professionals are also discovering that there is a rapidly expanding array of SaaS-based IT management solutions available which are making it easier for IT organizations to perform their day-to-day responsibilities.
Unlike Gartner’s survey research, THINKstrategies’ customer surveys have consistently found SaaS satisfaction levels, renewal rates and willingness to recommend at or above 90% over the past three years.
Ironically, a week after Gartner issued its SaaS survey findings it announced that the worldwide market for customer relationship management (CRM) applications grew 12.5% last year, fuelled by a 33% jump in SaaS-based solutions, which now represent 20% of the CRM market.
There is no question that as the SaaS industry grows, service quality will become diluted and customer expectations will vary more broadly. So, there is nothing wrong with warning customers and SaaS vendors alike to remain vigilant. I issued my own warning back in 2007. And, I continue to be concerned about the quality of support in the SaaS and broader cloud computing market.
But, when these warnings border on an indictment of the overall quality of SaaS solutions and the viability of this movement, it does a disservice to a marketplace which is fundamentally changing the way organizations leverage software to achieve their business objectives and the measurable business benefits they are experiencing.
Subscribe to my feed
2 things about this ‘report’:
1. The author mentions later “compared to what” which is what I am more curious about knowing. Overall of course companies are not in a ‘good mood’ and are not satisfied with IT delivering value (lack thereof) and the respondents probably feel like it’s another marketing pitch, same old, same old.
2. Many organizations I have spoken with there is a DEFINITE undercurrent of FEAR about SaaS. The most primal reaction in IT is that “my job could very well be eliminated” if we adopt SaaS. Thus empire builders and resume padders are terrified…their ‘value’ or skills will not be required in the new SaaS world, so best to strike it down now before the CIO hears about it.
I believe we have to talk to the CIO from a value, bottom line approach.
If you made and sold buggy whips, would you want your customers to know about the horseless carriages?
Paul — July 17, 2009 @ 11:13 am
I read the entire Gartner report, and feel as if the press release writer wasn’t exactly in sync with the report.
That said, I’ve seen a number of other surveys that say the same thing, including some I’ve done myself. I work for a company that was considering a SaaS pricing model for a new product.
We have changed that based, in small part, on the Gartner report and in larger measure from a pilot program with 10 of our best current customers. Loved the product, didn’t want to pay for it in monthly subscription fees — happy to pay for results, or on a purchase-price basis.
MrsH — July 17, 2009 @ 12:26 pm
Well said Jeff! Clearly you’re in the thick of things. Another point to keep in mind is that SaaS, depending on segment — both industry and application, is still in it’s early phases. As the services cross over from visionaries and early adopters, who are willing to deal with imperfections, to the mainstream where expectations are higher in terms of feature/functionality, there will be some tension. Eventually SaaS offerings will deliver better availability, responsiveness, and feature/functinality than competitive on premise apps . . . if they are not already. The pattern of new, disruptive technology is playing itself out in SaaS as it has in many other markets. CRM SaaS market penetration of ~15% illustrates the first vertical achieving critical mass. Others are following. Gartner will figure it out eventually. Unfortuntely, they don’t seem to be as innovative and understanding of innovation as they have in the past as far as I can tell from my experiences with them the last five years or so.
Ray — July 17, 2009 @ 4:52 pm
Nice post Jeff. You’ve come up with many new insights on this Gartner survey. My biggest issue with the survey and its analysis is that Gartner makes no attempt to define SaaS. I blog about this topic here: http://community.service-now.com/blog/rglauser/1879
These days it means so many things to so many people. I liken these results to a survey that concludes, “Drivers are underwhelmed by their cars.” What drivers? What cars?
Like you mention, understanding Gartner’s target market there is a good chance that most of the people surveyed have only experienced “SaaS” from one of the legacy vendors. If this is the case, I’d fully imagine they were underwhelmed.
Gartner also recently came out with Hype Cycle for Cloud Computing. SaaS is making making its way out of the trough of disillusionment up the slope of enlightenment despite the best efforts of the legacy vendors.
Rhett
Service-now.com
Rhett Glauser — July 18, 2009 @ 11:56 pm
Jeff – I would agree that some of the decrease in SaaS satisfaction may be reflective of the fact that IT is now playing a much greater role in the selection, deployment and management of SaaS applications than it has in the past. In a recent survey here at Conformity we found that IT is now involved in management and administration of SaaS applications in 72% of multi-SaaS organizations. At the same time, most SaaS applications today don’t easily integrate into the processes and tools that IT uses to manage their environments (directory services for example), which may be part of the ‘technical considerations’ factors that are picked up by the Gartner survey .
It’s probably not surprising that IT satisfaction may be lower, but at the end of the day it reflects ‘growing pains’ as SaaS becomes more mature. Much of the IT concern that we’re hearing is driven by integration and manageability issues that should be addressable by the SaaS industry as it becomes more familiar with enterprise IT requirements.
Scott Bils — July 20, 2009 @ 11:22 am
Jeff,
Great post. I wonder what the Gartner survey would be of satisfaction of on-premise applications? I think it depends on the application and who you ask, as many of the commenters have stated. I also find it funny that people complain about lack of features in SaaS when most of us barely use 10% of the features of local software. A good example is Microsoft Office. If Office only had in it what I actually use, it would be a lot smaller and a lot cheaper. I also agree with an earlier comment about Gartner and its mission to serve it customers rather than be the objective analysts they once were. I feel that an organization that takes 6 months to analyze a survey is a bit behind the curve.
Ron Arden — July 21, 2009 @ 8:20 am