Clusterix Inc. may offer the "Rolls Royce" of database sharding technologies, but its product simply wasn't the...
right fit for application performance management (APM) company AppDynamics.
Instead, a lengthy product evaluation led the growing San Francisco-based APM provider to choose ScaleBase for its database sharding needs, in part because of the company's willingness to provide a high level of personalized attention and support, said Boris Livshutz, server data lead at AppDynamics.
Clusterix's product was too expensive and offered several luxuries that AppDynamics could live without, Livshutz said.
"They store everything on [solid state drives], for example, and SSDs are very expensive compared to regular hard disk," Livshutz said. "Also, the wiring [they use] is all InfiniBand versus regular Ethernet, and we just didn't have that performance need."
Database sharding is a form of partitioning that allows users to horizontally scale their implementations without having to continually buy new hardware. The technology breaks up individual databases into smaller, more easily manageable pieces -- or shards -- and runs them across distributed servers.
For more on database sharding and database scalability
Learn about the scale-up vs. scale-out debate
Read about sharding relational databases in the cloud
Get to know Microsoft's sharding technology
The phrase database sharding was coined relatively recently by engineers at search engine giant Google; however, the concept of horizontal database scaling has been around for many years, according to published reports.
AppDynamics, which monitors both cloud and on-premises applications for companies in many industries, including Netflix, also felt that Clusterix's approach of offering an appliance -- a combination of hardware and software -- might lead to less flexibility and vendor lock-in.
"It's the hardware that really makes it expensive, and you can't really negotiate that stuff," Livshutz said. "They had what they had and they weren't really willing to play with the architecture for us."
CodeFutures' dbShards offering also misses the cut
AppDynamics first began looking into database sharding technologies about a year and a half ago, when it became evident that the company needed to scale its MySQL implementation with the needs of its customers -- and that continuing to throw hardware at the problem would eventually become prohibitively expensive.
"From the founder all the way down, everyone had been asking for an architecture that can scale, and our customers had been asking for an architecture that would scale because they would see performance problems," Livshutz explained. "They want to instantly scale and they want to make sure that we can instantly scale along with them."
In addition to evaluating Clusterix's offering, Livshutz and his team considered creating a database sharding system internally. They also took a close look at CodeFutures Corp.'s dbShards software.
The team decided that building the system internally would be a mistake, mainly because the small IT staff at the four-year-old startup lacked the expertise to maintain the system over the long term. Besides, Livshutz said, the team's time is better spent focusing on AppDynamics' core monitoring application.
"We can build it but who is going to run it?" Livshutz said. "That's not our bread and butter."
Livshutz found that the CodeFutures dbShards team was very helpful and willing to work closely with AppDynamics. But Livshutz was a little turned off by the company’s business model. CodeFutures, he said, is essentially a professional services firm that offers some software.
"With us being a startup and us being a software vendor, it made more sense to partner with someone that looked like us," he said, "someone that was more compatible."
AppDynamics eventually chose ScaleBase's Data Traffic Manager to manage and scale its database environment and has been gradually implementing and expanding the scope of the product for several months.
"They didn't have all the features we needed for the long term," Livshutz said of ScaleBase. "But they were very committed to making it successful and adding whatever needs to be added, which is what I look for in a [vendor]."
A monitor for the monitoring company
Livshutz said ScaleBase's features have made it relatively easy for AppDynamics to get into the database sharding game. Looking ahead, the company plans to build or buy a monitoring system for its database sharding operation.
"Basically, we'd like to have a rich monitoring environment just like the one we provide for applications," he said. "I think that [would be] a great feature for sharding."