NS0-901 AI Software Architectures

AI Software Architectures Detailed Explanation

1. Core Components

AI software architecture is like the “blueprint” or structure of how an AI system is designed. It includes all the tools and platforms needed for:

• Preparing data

• Training models

• Deploying models

• Monitoring models in production

Data Pipeline Tools

What they do:

• Move, transform, and schedule data tasks across different systems.

• Prepare data for training or real-time predictions.

Common tools:

• Apache Kafka: Handles real-time data streams. For example, it might take live click data from a website and send it to the training system.

• Apache Airflow: Manages workflows. You can schedule tasks like data collection, cleaning, training, and reporting.

Why important:
AI needs lots of clean and timely data. Data pipelines automate and organize that process.

Training Platforms

What they do:

• Allow data scientists to train machine learning or deep learning models.

• Provide features like distributed training, GPU usage, experiment tracking, and job management.

Common platforms:

• Kubeflow: An open-source platform built on Kubernetes for ML workflows.

• MLFlow: Lightweight and easy-to-use tool for managing the ML lifecycle.

• Amazon SageMaker: A cloud-based tool that offers a complete training and deployment solution.

Why important:
Training complex models requires powerful infrastructure and efficient job scheduling — these tools make it manageable.

Serving Platforms

What they do:

• Take a trained model and make it accessible to applications (like mobile apps or websites) through an API.

Common tools:

• TensorFlow Serving: High-performance tool to serve TensorFlow models.

• NVIDIA Triton: Supports multiple frameworks (TensorFlow, PyTorch, ONNX). Optimized for GPU-based inference.

Why important:
You don’t just train a model — you also need to let others use it easily. Serving platforms make models “live.”

2. Experiment Tracking

What it is:

• A system to record, manage, and compare different training runs of an AI model.

Key elements tracked:

• Model version

• Hyperparameters (settings used to train the model)

• Training data used

• Performance metrics (accuracy, loss, etc.)

Common tool:

• MLFlow:

  • Logs experiments

  • Saves models and metadata

  • Supports model comparison

  • Integrates with many frameworks (TensorFlow, PyTorch, etc.)

Why important:
Without tracking, it’s hard to know:

• Which model version worked best

• What parameters led to success or failure

• How to reproduce a good result

Example:
A data scientist runs five versions of a fraud detection model. With experiment tracking, they can go back and see which version gave the best balance of precision and recall — and why.

3. Containerization and Orchestration

These technologies help package, deploy, and scale AI models in a consistent and efficient way.

Docker (Containerization)

What it does:

• Packages an AI model with all the code, libraries, and system settings it needs to run.

• Creates a container — a portable unit that works the same on any system.

Why use Docker:

• Avoids “it works on my machine” problems

• Ensures consistent environments across teams

Example:
You create a Docker container with Python, TensorFlow, and your trained model. You can now run it on a laptop, server, or cloud.

Kubernetes (Orchestration)

What it does:

• Manages multiple containers

• Automates deployment, scaling, and recovery of AI services

Why use Kubernetes:

• Runs AI workloads efficiently across many servers

• Can scale up when there’s high demand and scale down to save resources

Example:
You have 100 users sending real-time requests to your model. Kubernetes can launch more copies (pods) of your model to handle the traffic.

4. CI/CD in AI (Continuous Integration / Continuous Deployment)

AI models change often. CI/CD helps teams test and release updates quickly and safely.

CI (Continuous Integration)

What it is:

• Automatically tests model code and data whenever changes are made

Why important:

• Detects bugs early

• Validates that models still work with new data

CD (Continuous Deployment)

What it is:

• Automatically deploys tested models to production or staging environments

Why important:

• Faster delivery of improved models

• Enables rollback if performance drops

Example:
A new version of your customer recommendation model is ready. CI/CD pipelines test it, validate it, and deploy it — all automatically.

5. Common Frameworks

These are software libraries used to build and train AI models.

For Deep Learning:

• TensorFlow: Google’s popular library for building deep learning models.

• PyTorch: Flexible and developer-friendly; widely used in research and production.

• Keras: Simplified API often used with TensorFlow; easy for beginners.

For Traditional Machine Learning:

• Scikit-learn: A classic library for non-deep learning models like decision trees, SVMs, and linear regression.

Why important:
These frameworks offer pre-built functions, performance optimization, and community support — making AI development much faster and easier.

AI Software Architectures (Additional Content)

1. Notebook vs Pipeline Architectures

AI development environments often use two different architectural patterns depending on the maturity of the solution:

Notebook-Based Development

• Tools: Jupyter Notebook, Google Colab, Zeppelin

• Purpose: Exploratory analysis, quick prototyping, and visualization

• Strengths:

  • Interactive and flexible

  • Ideal for early-stage experimentation

  • Easier for individuals or small teams

• Limitations:

  • Poor version control and reproducibility

  • Hard to scale or automate

  • Not ideal for production environments

Pipeline-Based Development

• Tools: Kubeflow Pipelines, MLFlow Projects, Airflow DAGs

• Purpose: Automate data ingestion, training, evaluation, and deployment as repeatable steps

• Strengths:

  • Reproducible and modular

  • Supports automation and scalability

  • Easy to integrate with CI/CD and MLOps

• Limitations:

  • Higher setup complexity

  • Requires pipeline orchestration tools (e.g., Kubernetes)

Summary:

Feature	Notebook	Pipeline Architecture
Use case	Prototyping	Production automation
Tool type	Interactive notebooks	Orchestrated workflows
Flexibility	High	Structured and rigid
Scalability	Low	High
Collaboration	Limited	Team-oriented and reproducible

2. Model Registry and Model Management

A Model Registry is a centralized repository to store, manage, and track machine learning models throughout their lifecycle.

Functions

• Store model artifacts and metadata

• Track versions and their performance metrics

• Transition models through lifecycle stages (e.g., staging → production)

• Enable rollback to previous versions if needed

Common Platforms

• MLFlow Registry:

  • Tracks model runs, versions, and stages

  • Integrates with CI/CD pipelines

• SageMaker Model Registry:

  • Integrates with SageMaker Pipelines and MLOps tools

  • Automates approval workflows and deployment

Benefits in Production

• Ensures consistency and traceability

• Reduces deployment risk

• Supports auditability and reproducibility

• Enables automated rollbacks when models fail

3. AI Software Architecture and NetApp Tools Integration

While NetApp product usage may not be deeply tested, awareness of its tools is relevant in NS0-901.

NetApp DataOps Toolkit

• Automates dataset versioning, cloning, and snapshotting

• Accelerates experimentation by rapidly provisioning consistent environments

• Reduces storage overhead through space-efficient cloning

BlueXP

• Manages multi-cloud and hybrid AI data infrastructure

• Supports policies for data mobility, compliance, and optimization

• Provides a control plane for AI-related data workflows across clouds

Kubernetes + NetApp Trident

• Trident is an open-source storage orchestrator that integrates NetApp storage with Kubernetes

• Enables persistent volumes for containerized AI workloads

• Used to provision, scale, and snapshot storage for AI training jobs

These tools strengthen MLOps workflows by providing storage scalability, data governance, and faster iteration cycles.

4. Multi-Framework Model Support and ONNX Optimization

AI model deployment platforms increasingly require flexibility in supporting multiple training frameworks and optimizing for inference speed.

ONNX (Open Neural Network Exchange)

• Purpose: Open standard to allow interoperability between AI frameworks

• Created by: Microsoft and Facebook

• Supports:

  • Exporting models from PyTorch, TensorFlow, Scikit-learn, etc.

  • Running models on multiple inference engines (ONNX Runtime, TensorRT)

Triton Inference Server

• Developed by NVIDIA

• Supports TensorFlow, PyTorch, ONNX, and TensorRT models

• Features:

  • Dynamic batching

  • Concurrent model execution

  • CPU/GPU target configuration

TensorRT

• NVIDIA’s inference optimization library

• Converts models (including ONNX) into highly efficient GPU executables

• Performs layer fusion, precision tuning (e.g., FP32 → INT8), and memory optimization

Why it matters:

• Reduces inference latency

• Enables mixed-framework deployments

• Essential for environments requiring high throughput at low cost