Azure-to-AWS migration for Producteca: a conversion layer that took Azure Search AI to OpenSearch without refactoring the microservices

Challenge

The project involved migrating various workloads from Azure to AWS, with one critical challenge: migrating the search indexes from Azure Search AI to OpenSearch despite the incompatibility between the two services. The main obstacles were the format incompatibility between the Azure SDK queries (ODATA) and the OpenSearch SDK queries (DSL), the differences in property mapping for operations such as grouping, filtering and ordering, and the difficulty of changing the query format across a large number of microservices in a limited timeframe. The hardest part was converting the Azure Search AI filters, which express operators, groupings, comparators and iterators within a single plain text string.

Solution

The solution was built on an abstraction and conversion strategy: an intermediate layer that translates queries and data between the two systems. On one side, Craftech adapted the mapping of the Azure Search AI indexes to OpenSearch —with the support of an OpenSearch expert provided by AWS—; on the other, it converted the queries from ODATA to DSL. Since the customer queried using the Azure SDK in Node.js through an intermediary class (with find, upload and delete methods), an equivalent class was developed that performs the conversion and queries via the OpenSearch SDK. For the filters, an API was built that exposes one endpoint per index: using a Model class equivalent to the mapping and a mock call to OpenSearch, it transforms the filter from a plain text string into a JSON object ready to be added to the query. To keep latency to a minimum, the converter API was deployed inside the same Kubernetes cluster where the microservices run.

Results

The transition from one cloud provider to another was achieved in a fraction of the time it would have taken to refactor the microservices, with significant savings in development hours and the ability to quickly capitalize on the benefits of the new provider. The architecture built on the conversion API and the intermediary class made the migration nearly transparent to the microservices, minimizing the risk of errors and the need for extensive regression testing. And the detailed documentation of the solution left internal teams a foundation to adapt their microservices gradually and in a controlled way going forward.