Machine Learning Data Pipelines
Categories:
Machine Learning
ML
Data
Data Pipelines
Data Engineering
Kafka
Kafka Streaming
Deep Learning
How do we move information realtime and connect machine learning models to make decisions on our business data? This presentation goes through machine learning and Kafka tools that would help achieve that goal.
In this presentation we start with Kafka as our data backplane, how we get information to our pub/sub. As they enter Kafka, how do we sample that data and train our model, then how do we unleash that model on our real time data. In other words, picture extracting samples for credit card approvals for training, then attaching the model for online processing: The moment we receive an application we can either approve or disapprove a credit application based on a machine learning model trained on historical data.