The ZoomInfo application team gives priority to existing tools and technology when developing new features and new products.
There are several data storage tools (data stores or databases) currently in use and the team looked to leverage these when choosing tooling for FormComplete (see this blog piece for more information about the tool selection process). Unfortunately, their initial choice did not meet the priorities of the project and had to be changed to another option that better fit the project.
What is FormComplete?
FormComplete is an optional add-on to the DiscoverOrg and ZoomInfo combined platform that allows customers to access ZoomInfo contact and related company data from forms on their own website. This helps their end users provide more detailed information prior to form submission. When an end user provides an email address ZoomInfo supplies data for the other form fields (if a match is found).
For more information about how FormComplete works, read this blog post.
Data Storage in FormComplete
FormComplete currently stores records of each form using the feature submitted on customer websites and the data that originated in ZoomInfo as part of that submission. These forms are designed by our customers and, other than a required email address field, may contain any of dozens of supported fields in any order desired by the customer designing the form.
The Initial Data Store Choice for FormComplete
Because the forms using FormComplete are flexible and may have any number of different fields, ZoomInfo initially decided using a NoSQL database to manage the related data made the most sense. Using this type of data store relying on series of key-value pairs means that there’s no need to define specific schemas where each table definition maps to the specific fields of each form created using FormComplete in order to store data originating from that form. The necessary key-value pairs can just be defined on the fly with much less overhead.
A popular NoSQL database, MongoDB, was already used by the ZoomInfo application team on other development projects. Given this, the team working on FormComplete decided to use MongoDB as well so they could leverage existing infrastructure and expertise.
Why ZoomInfo Changed the Data Store Used by FormComplete
In addition to the direct FormComplete data stored in its dedicated database instance, the FormComplete processing requires complex queries and joins with other data stores including the main person and company data stored in Apache Solr.
The data ZoomInfo stores for FormComplete is used to populate customer dashboards in the combined DiscoverOrg and ZoomInfo web application displaying the data submitted with each form as well as some metadata about the searches. Unfortunately, retrieving this data and joining it with related data stored in other ZoomInfo databases was too slow. Providing fast data access in these dashboards was identified as the highest priority for FormComplete and, unfortunately, MongoDB was not meeting the product goals in this area.
Using MySQL as the FormComplete Data Store
After determining that MongoDB would not meet the highest project priority – that initial form processing be extremely performant – the application team looked at the other databases in use at ZoomInfo and decided to switch to a SQL database, MySQL.
SQL is designed for quick database joins and, because it uses predefined schemas, can usually handle complex queries fairly quickly. This comes at a price – scalability. When using MySQL, every single customer website form using FormComplete must have its own table defined by a set schema created during the initial FormComplete configuration for the form. As more and more forms are created and there are more and more schemas defined in the data store, performance will likely decrease and maintenance and management of the data becomes more difficult. With the current usage, this is not a problem but it may become an issue at some point in the future. At that time, another new database solution that offers performance that is good enough while being scalable enough to handle the required form load may be required.
Final Thoughts
The ZoomInfo application team made an initial decision to try the NoSQL database MongoDB for storing FormComplete submission data, guessing that it was the best tool available from those already in use. They discovered that they were wrong about this because it failed to meet the project’s highest priority (performance) and the team quickly changed course to the SQL database MySQL. This is not a perfect solution and it may be necessary to make another database change in the future.
ZoomInfo is absolutely okay with both of these things. Trying a solution that doesn’t work then quickly changing to a better option is considered a better process choice than spending gobs of time trying to find the perfect answer – which may not exist – up front. In this case the perfect may, indeed, be the enemy of the good enough and the fear of failing to do the right thing may have set the project release back considerably. Instead, we have a working product that meets our current customer needs.
