Lessons Learned from Adopting a First Microservice – Technology

ZoomInfo decided to break its search service away from the main monolithic application and make it a microservice used by that application and, potentially, other ZoomInfo microservices or applications.

See this blog post for a discussion of why and how we did this. All told, we’re happy with that decision, but it definitely required some adjustments on both the technology and the process side. This article will provide an overview of the benefits and challenges ZoomInfo identified with technology while adopting the search microservice.

Technology Benefits

The technology benefits from adopting microservices fall into three general buckets:

1. Choice
2. Flexibility
3. Upgrades

These are really different sides of the same coin, all emanating from the idea that each microservice is an independent black box application to all other code in your larger ecosystem. This gives the folks working on each microservice the freedom to choose the tools and technology that best suits their specific needs without worrying at all whether other folks at the company are interested in or ready to use them. 

Work in a Java shop but want to use Python for a specific feature? No problem, and no need for all of those folks using Java to change a thing to work with your Python microservice. They won’t even know it’s not using Java unless you tell them or share your code. Similarly, consumers of a service shouldn’t care about your internal architecture – you can use Kubernetes or Docker Swarm or AWS orchestration tools or even choose a non-traditional architecture that doesn’t use containers at all if that seems to best suit your needs and it’s all transparent to the other developers in your company provided you meet your contractual obligations as published (support the published interface rules and meet any promises around performance, mostly). ZoomInfo uses Kubernetes with a bunch of supporting tools, but that could always change in the future without impacting how the service works as far as the folks using it are concerned.

It also means that new tools and new technology that do have wider interest can be tried out on a small scale first. There’s no longer the need to commit the entire development team to a big migration or experimentation that may fail and need to be widely cleaned up. If your organization is using Java across the board but some folks have suggested they may get better results if they switch to Kotlin to generate the same bytecode, try it out in one microservice first and see how it goes. If it’s a smooth transition, then slowly roll it out to more and more development projects. If it proves to be more difficult than expected or if it’s not viable only the folks working on that single microservice feel the pain. They may have to come up with a new solution on the fly or scrap the new technology entirely. Either way, no one else is affected at all. ZoomInfo looks at its experimentation with creating a search microservice in this light. The original architecture used Kubernetes for both orchestration and load balancing. However, there were issues with the resulting load balancing (see this blog post for more information) requiring a quick adjustment of plans. A new solution using Istio was found and we were able to move forward. Now that the search microservice has been successfully deployed with the reconfigured architecture, other features may be broken away from the monolith and completely new features may be developed using microservices instead of being added to the monolith in the future.

In addition to trying new programming languages, libraries, helper applications, and the like, most microservice environments can be easily moved to new cloud environments without much fuss. Want to switch from Azure to AWS? Microservices deployed inside containers make that much, much easier. ZoomInfo hasn’t tried to do this type of migration yet, but designed the search microservice to make this as easy as possible just in case it’s something we want to do in the future.

Making upgrading to new versions of tools, libraries, and helper applications as easy as possible was a primary driver of ZoomInfo’s move to start using microservices. We often fell one or even several versions behind on Apache Solr, the technology we use to power search, because upgrading within the monolith was so onerous it brought all new development work to a halt for months. This meant we weren’t absorbing bug fixes and didn’t have access to new features. With a search microservice we don’t have to worry about all of the many places within the monolith calling search, but rather just on upgrading the microservice. As long as the published interfaces and APIs remain the same (or at least completely backward compatible), this upgrade has no affect at all on any applications calling the search service. If new functionality is added with corresponding additions to the interfaces, the search consumers could adopt them as desired within their own scheduled development cycles with no regard to the schedule of the search release adding support for those features.

Technology Challenges

Adoption of microservices brings with it some technology challenges too. They include:

1. All technology around microservices is relatively new
2. New technology and tools are constantly available
3. Finding experts or getting support on very new tools is difficult

These issues all stem from the relative newness of microservices. Microservices haven’t been around for very long and the technologies used to make them work don’t have decades or even years of adoption and refinement. The number of folks with significant experience with some of the key technology is small compared to the number of people who know how to code in Java or query against SQL databases or use APIs. If you want technology that’s been widely used for a long time before allowing its use in your production environment, microservices are not for you.

Further, you need to factor in that new tools and technology are being actively developed at a fast pace. While the most popular container and cloud tools have been around for a number of years now, many other related tools have not. The right tool for a specific component of a specific project might be brand new with limited or no formal support availability and only a handful of users posting about or responding to issues online at sites like Stack Overflow or similar places. This can be frustrating and slow down the development process when something goes wrong or someone can’t figure out how to implement something important.

For example, ZoomInfo currently uses Istio for load balancing and to add a few other features it supports that add value to the search service (more information about how we use Istio is available here. Istio was first released in a very limited form in 2017 and a true production version (1.0) has only been available since the summer of 2018, well less than a year before we started creating a search microservice. It’s added a lot of value to our stack but it was also difficult to find answers to questions when we needed help figuring out how to do specific things or debugging issues that came up because very few people are using it yet. On the whole, the positives outweigh the negatives, but that doesn’t make the negatives any less problematic.

One way ZoomInfo overcame these difficulties is via a service contract providing support for cloud services. Because we use the versions of Kubernetes and Istio supplied by our cloud provider, we were able to get some help when Istio issues cropped up. Such a contract may seem like an unnecessary expense, but it may be worth the cost if you plan to use some of the newer technology without other obvious support routes.

Final Thoughts

When trying a new development model such as microservices, people often have to use new technologies that aren’t as mature as some of the other technology they’ve been using in the past. This can present some challenges, particularly in finding the right set of technology to use and in getting the necessary technical support for those technologies. Overall ZoomInfo is pleased with our initial experiment with microservices but there were some bumpy moments to overcome.