Complex software architecture for intuitive video personalization

A leading insurance company offers tailored explanatory videos for its corporate customers in the area of car fleet insurance. The content of the videos should not only result from contract data, but should also allow personalization by insurance agencies themselves. Accordingly, the requirement was based on a complex software architecture with several interfaces, which the developers at Scandio were able to implement successfully and in close coordination with the customer and other service providers.

One of the largest German insurance companies was looking for a solution for the car fleet insurance sales team to provide their customers individually tailored explanatory videos on contracts. After the conclusion of a fleet insurance policy, contract details such as the name of the insured company, the number of insured vehicles or additional insurance components should be aggregated from the internal backends of the insurance company. The insurance company's agents should also be able to customize the videos for their customers on top of that.

The collected data is then processed in a backend and sent to an external service provider, who renders the personalized onboarding video from it. Finally, the customer receives a link to a landing page that displays contract details and other information in addition to the video.

Challenge

The project to personalize the car fleet onboarding videos is the third project in this context, the first two explainer video projects having been developed without Scandio's involvement. In the most recent case, Scandio developers were entrusted with developing the underlying software architecture and implementing the required components in both the front-end and the back-end.

Also in the third project, there are various ways in which the insurance data can influence the finished video:

• For one thing, the data can change the flow of the video: certain insurance components are only explained if they are also covered in the contract.
• For another, the audio can be customized in various ways, so that, for example, a general salutation such as "your company" is replaced by a more specific one such as "your craft company". Visual customizations such as rendering a text at a specific point in the video are also possible.

The big difference to the previous video projects:  Differently than the previous onboarding videos, there is now the additional option of individual personalization by the insurance agent. For example, agents can record a personal welcome video, insert the contact person name, and promote individual agency services (e.g., a vehicle registration service) in the video. This additional personalization is possible for insurance company employees via a frontend developed by Scandio.

Similar to the previous two video projects, it was requested to use Spring Boot with Kotlin for the backend and Angular with TypeScript, as well as the insurance company's in-house UX library for the frontend. The advantage of using consistent technologies is less training time and increased potential for reusability of existing solutions. An Amazon Aurora PostgreSQL database was provided. A parallelizable operation of the application with multiple instances was also part of the requirements to allow for easy scaling and blue-green deployments.

The most important task and also challenge was the design of the project's software architecture and the communication with those responsible for the neighbouring systems. Especially when designing the software architecture, we had to interact with many external components and also consider quality requirements of the customer when designing the system architecture. Through extensive coordination with the external stakeholders, technical best-practices and clear prioritization, we were able to meet the defined requirements to the customer's complete satisfaction.

Benedikt Hunger, Developer at Scandio

Solution

Preliminary work on the part of the insurance company
In preparation, the frontend was designed as a clickable mock by a UX designer in close collaboration with business analysts from the insurance company as well as external service providers, and coordinated with the insurance company representatives (who would ultimately work with it). Likewise, the planning of the video's flow and personalization logic, as well as the creation of the video, happened independently of Scandio's work.

Planning the project architecture and later adjustments
The first concrete step related to the project was planning the project architecture. In the first iteration, the architecture contained the following components:

• the frontend that insurance agents work with (consisting of different pages)
• the customer landing page with embedded video player
• the backend for video creation
• an application database
• an AWS S3 bucket for storing and delivering the videos
• various external components

The external components include the service provider for creating the videos, the internal systems for retrieving the contract details, the insurance application process as a trigger for video creation, and a database as an interface for sending e-mails.

In addition, the Scandio employees decided to implement an additional backend that would be used only for the upload of the personal welcome videos by insurance employees. The reason was to avoid an impact of potential upload issues with the rest of the process of creating and delivering the onboarding videos. Such upload issues might arise from the large amount of data in a video stream. An additional benefit to this approach was that upload and video rendering could be scaled independently if, for example, there were a lot of video uploads at any one time.

In the course of development, various adjustments to the architecture were necessary. In particular, the introduction of a new cloud infrastructure required the application to be split into three zones: one for the interfaces to the outside world, one for the interfaces to the internal network, and one for the business logic. This made some of the planned communication paths impossible, so additional components became necessary. This resulted in the following challenges especially:

• The trigger to initiate the video creation process had to run through an Apigee API gateway provided by the customer. But, the API gateway did not have direct access to the backend, so an additional proxy was needed here, for which the team used an NGINX reverse proxy.
• Access to the email personalization database was not possible directly from the backend. Therefore, the Scandio team provided its own mailing service for this purpose, which is an API for using the email database and is now also used by the existing video services. Another advantage of this approach is the easy exchange of the email provider, should that be necessary in the future.
• To ensure secure communication between services, each connection via mTLS had to be secured on both the client and server side. To ensure this, the Scandio developers used provided Istio sidecars. This in turn initially led to problems in connection with the corporate proxies in the direction of the internal services still running on old cloud environments. This was because the mTLS configuration had to be switched off again in order to establish a secure connection via the proxy using HTTP Connect. For this reason, communication to the proxies had to be explicitly disabled in the configuration of the Istio sidecars, which fortunately was easily possible using a Kubernetes custom resource.

An agile way of working meant that the necessary adjustments to the architecture could be made flexibly.

Offloading shared code to a library
Since this was the third onboarding video project, Scandio experts offloaded common code to a Kotlin library in preparation to reduce code duplication. This shared code contains the logic for creating the video and for verification and error handling in this creation process. Furthermore, the requests to render the video and a regular polling of the rendering status are part of the newly created program library. It also verifies that an object with the expected name and a reasonable file size has been successfully placed in our S3 bucket.

Similarly, thumbnails are rendered for the personalized email. Also included in the library is logic to lock individual videos for processing to ensure parallelized operation of our backend with multiple instances.

Development of core functionality without agent personalization
After planning the underlying architecture, the Scandio team started to develop the customer landing page and the backend. For video creation, Scandio's backend staff was able to draw on the previously developed library, which they now further adapted to the new requirements in parallel.

Additional help with the agency frontend
In parallel with the development of the new video personalization service, Scandio's developers also performed maintenance and support tasks on existing systems. Mainly, this included the requirement to migrate the existing application database to the new cloud environment. This, along with relatively complex knowledge to acquire around the new cloud environment, implied that the completion of the project within the planned timeframe was in jeopardy. To avoid too many delays, Scandio provided short-term support for one month with three additional developers for the development of the video personalization.

Here, of course, it was particularly important to avoid long training periods. It was not expected that the additional employees would be given access to the internal systems at such short notice. For this reason, the Scandio team decided, in preparation, to precisely specify the interfaces to the backend and encapsulate them away in such a way that development of the agency frontend for video personalization was possible locally without access to the backend systems. A clone was then created in Scandio's internal Bitbucket for the frontend repository. From that point on, the developers used the state in Scandio's internal Bitbucket as the source of truth and regularly synchronized it with the upstream repository in the insurance company's corporate Bitbucket. This was possible with almost no additional effort, and it also allowed the resulting code to quickly test in integration with the backend.

While the core team took care of finishing the backend, the agency frontend was almost fully developed by Scandio's support team. Last details that required a stronger integration into the overall system - like the upload of the greeting video snippet - were added in a second step.

Pentests and test sessions
Due to the short-term reinforcement of the team, a pentest of both the customer and the representative frontend could take place with only small findings. Joint test sessions also revealed a few bugs, but these were quickly resolved. Finally, the agency personalization still had to be tested and the agency frontend had to undergo an accessibility test in order to comply with guidelines of the staff council. During all these tests, the Scandio development team provided support and made final adjustments.

Result

Thanks to the work of the Scandio development team, insurance agents can significantly strengthen sales and customer loyalty through explanatory videos tailored to the individual customer. In addition to useful standard settings, the developed solution also offers the possibility of customization for the respective agency (such as offering a service for motor vehicle registration), up to an individually personalized approach to the customer. By separating responsibilities both in the various backend systems and in the frontends for end customers and agents, it was possible to ensure good maintainability and extendibility of the overall system. The solution for personalizing the onboarding videos has been in use since March 2023.