Smart Cooking: Get your baking time always right with AI

Ovens of BSH Home Appliances, a global leader in the home appliance industry, receive forecasts on their remaining baking duration from AI models served in the cloud. But getting the AI to handle thousands of customer requests and provide accurate time predictions was very complex. So, for developing the remaining time forecast feature for baking sensor and meat probe applications, multiple fields of expertise collaborated - including a key team of data scientists and developers from Scandio for infrastructure and machine learning.

Novel ovens manufactured by BSH come with Artificial Intelligence (AI), providing users with an unprecedented smart cooking experience: Place your food in the oven, press play, and the oven will tell you when it's finished - no matter if you are in the kitchen or not since the AI shows the remaining cooking time via the oven display or the Home‌ Connect smartphone app. The cloud-based feature does not stop at cakes; it also includes a meat probe ensuring perfectly prepared roasts.

But as simple as the remaining time forecast feature is for the user, the technology behind it is complex. Developing the AI to handle thousands of customer requests and provide accurate time predictions on both the oven display and app required significant work and expertise.

The Challenge

The challenge in offering an AI-powered baking and roasting feature for BSH's advanced ovens lies in solving the temporal problem of predicting the baking time with high accuracy. This requires a constant connection and feedback between the AI model in the cloud and the oven during the baking process, taking into account specific factors for different baking dishes. For the AI the baking process for a pizza differs fundamentally from preparing the Thanksgiving turkey.

Furthermore, the feature must be able to handle parallel prediction requests from thousands of ovens simultaneously and be scalable for varying loads and peak times like Christmas evening (when a lot of people prepare dinner simultaneously) vs. a regular weekday morning.

The feature should also be available in multiple countries, compatible with legacy, recent, and future oven devices with different firmware and connection modules, and provide an administrative panel and dashboard for BSH stakeholders and data scientists to monitor and optimize the model's accuracy. Additionally, relevant model output data must be stored to enhance the model's accuracy.

The Solution

The remaining time forecast feature for new ovens was developed by a team of 10 developers and data scientists from Scandio within a little over a year, using cutting-edge programming tools and technologies.

The collaborative effort between multiple fields of expertise made the remaining time forecast feature such a success. As key interplays, one can highlight the following:

• Development efforts that connected the ovens with the cloud's business logic and data storage.
• Machine Learning Operations (MLOps) that served and connected the AI models to the business logic.
• Data scientists provided a custom and highly accurate model and iteratively improved it meaningfully.

The experts chose to write microservices with Python for AI proximity. To develop frontend-related services for administrative purposes of stakeholders, TypeScript, Grafana with Prometheus as well as Datadog were used. The service and process metrics, plus selected live data from the model, are available on a custom dashboard and an admin panel UI. This does not only depict the state of health, but also allow stakeholders to feel the readiness and state of the feature.

Service orchestration is done via a Kubernetes cluster within AWS, allowing the team to maintain flexibility and avoid vendor lock-in, while also permitting them to compare and optimize their solutions with industry standards and future code solutions.

The infrastructure was set up using Terraform, with its Infrastructure as Code principle, enabling the team to manage the infrastructure with version-controlled code for testing and production environments. This approach also made it possible to hand over parts of the feature, reducing the need for revisiting tasks, and allowing for scalability as the customer base grows.

Machine learning models and pipeline inferences were set up for each supported dish for efficient and reliable finish time prediction. The models were trained using TensorFlow, and the pipeline inferences were developed using Seldon Core.

These tools are widely used in the industry for AI jobs and receive regular updates for improvement and for keeping up with state-of-the-art practices.

The connection between the oven and the backend was established through a custom event-triggered architecture. Based on the oven type and selected baking dish, a unique process is created in the backend, responsible for time-based baking predictions. A streaming connection constantly routes the oven measurements to the required AI model and sends the model output back to the oven several times per minute until the dish is done. The event streams for the data flow between the model and oven were set up using Apache Flink within Kinesis Data Analytics and Apache Kafka.

To ensure scalability, the team implemented a concept that easily handles the naturally growing customer base with the increase of smart households.

The data from oven measurements and model output are persisted to allow Data Science teams to improve the models further. The data is backed up and privacy engineering is applied to ensure GDPR compliance.

The cross-functional teams chose these approaches and technologies for their proven reliability, scalability, and their means to handle the demands of this complex AI solution. As a result, Scandio was able to provide BSH with a high-quality AI solution that meets the company's requirements and successfully drives its business.

Outcome and Results

As an owner of a smart oven, you'll no longer have to guess when your food will be ready. Instead, with the AI-powered remaining time forecast feature, you'll know exactly when your dish reaches the perfect result. Thanks to Scandio, BSH was able to launch this innovative feature for its newest ovens on time in January 2023.

The remaining time forecast is now an essential feature of the BSH Home Connect app, making it one of the first AI-enabled features offered by BSH to its customers. The solution is a clear testament to BSH's innovative power and vital differentiator in the market of home appliances.

"For Scandio, this project marked the start of a successful and popular AI segment. We're incredibly proud to have been a part of this groundbreaking project, and to have delivered a feature that has improved the lives of smart oven owners around the world."

Marco Michatz, Software Engineer at Scandio