Each task in an ML pipeline is defined using relevant component(s) in the TFX library, and they get executed with a pipeline orchestrator such as Airflow or Kubeflow Pipelines.
In this step, a data file is read or requested from an external service for the pipeline to run. Before passing the ingested dataset to the next component of the pipeline, the available data is converted into TFRecord files.
The data files can then be validated using TensorFlow Data Validation (TFDV), a python-based package in TFX, where some of the critical tasks like checking data anomalies, consistency of schema, statistical alignment of new datasets with training datasets can be performed.
In this step, using TensorFlow Transform (TFT) components the features are converted into consistent numerical representations so that the machine learning model can be trained with the numerical representations of the features.
With the preprocessed data ready, the Trainer component in TFX is used to train ML models either as Keras or Estimator models.InTFX Trainer component, ML training defined as training steps and not as epochs.
In a TFX pipeline, TensorFlow Model Analysis (TFMA) calculates metrics based on the saved model that is exported by the Trainer component. TFMA is used to monitor the performance metrics through time—before deployment, after deployment, and while in production.
TensorFlow Serving (TFS) is a package to deploy any TensorFlow graph and to make predictions from the graph through its standardized endpoints. TensorFlow Serving handles the model and version management, serves models based on policies, and allows to load the models from various sources.
While building a machine learning pipeline, we often try to answer complex business questions or model a complex real-world system. One single metric is often not enough to identify whether a model will answer the question accurately.
With TFMA, detailed, visual metrics can be generated for different models on a time series, also slices of datasets can be assessed for performance.
A go-to tool that offers monitoring capabilities for all ML stakeholders at all value-creation points of ML lifecycle, and that fully leverages Tensorflow's data validation and model evaluation capabilities. Provides a neat-and-clean, and modular observability experience to keep the ML model delivering the right value to the user.
Anodot’s MLWatcher helps to generate and monitor metrics from ML models in production. Being an open-source Python agent, it allows setting the threshold for metrics based on business requirement
Firebase’s mobile and web application capability leveraging the Tensorflow Lite platform means a lot for the increased need for monitoring production ML model performance.
Please feel free to schedule a demo for understanding how Qualdo does ML model monitoring rapidly