book

Designing Machine Learning Systems

by Chip Huyen

May 2022

Intermediate to advanced

386 pages

12h 25m

English

O'Reilly Media, Inc.

Book available

Read now

Unlock full access

Who This Book Is ForWhat This Book Is NotNavigating This BookGitHub Repository and CommunityConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments
When to Use Machine LearningMachine Learning Use CasesUnderstanding Machine Learning SystemsMachine Learning in Research Versus in ProductionMachine Learning Systems Versus Traditional SoftwareSummary
Business and ML ObjectivesRequirements for ML SystemsReliabilityScalabilityMaintainabilityAdaptabilityIterative ProcessFraming ML ProblemsTypes of ML TasksObjective FunctionsMind Versus DataSummary
Data SourcesData FormatsJSONRow-Major Versus Column-Major FormatText Versus Binary FormatData ModelsRelational ModelNoSQLStructured Versus Unstructured DataData Storage Engines and ProcessingTransactional and Analytical ProcessingETL: Extract, Transform, and LoadModes of DataflowData Passing Through DatabasesData Passing Through ServicesData Passing Through Real-Time TransportBatch Processing Versus Stream ProcessingSummary
SamplingNonprobability SamplingSimple Random SamplingStratified SamplingWeighted SamplingReservoir SamplingImportance SamplingLabelingHand LabelsNatural LabelsHandling the Lack of LabelsClass ImbalanceChallenges of Class ImbalanceHandling Class ImbalanceData AugmentationSimple Label-Preserving TransformationsPerturbationData SynthesisSummary
Learned Features Versus Engineered FeaturesCommon Feature Engineering OperationsHandling Missing ValuesScalingDiscretizationEncoding Categorical FeaturesFeature CrossingDiscrete and Continuous Positional EmbeddingsData LeakageCommon Causes for Data LeakageDetecting Data LeakageEngineering Good FeaturesFeature ImportanceFeature GeneralizationSummary
Model Development and TrainingEvaluating ML ModelsEnsemblesExperiment Tracking and VersioningDistributed TrainingAutoMLModel Offline EvaluationBaselinesEvaluation MethodsSummary
Machine Learning Deployment MythsMyth 1: You Only Deploy One or Two ML Models at a TimeMyth 2: If We Don’t Do Anything, Model Performance Remains the SameMyth 3: You Won’t Need to Update Your Models as MuchMyth 4: Most ML Engineers Don’t Need to Worry About ScaleBatch Prediction Versus Online PredictionFrom Batch Prediction to Online PredictionUnifying Batch Pipeline and Streaming PipelineModel CompressionLow-Rank FactorizationKnowledge DistillationPruningQuantizationML on the Cloud and on the EdgeCompiling and Optimizing Models for Edge DevicesML in BrowsersSummary
Causes of ML System FailuresSoftware System FailuresML-Specific FailuresData Distribution ShiftsTypes of Data Distribution ShiftsGeneral Data Distribution ShiftsDetecting Data Distribution ShiftsAddressing Data Distribution ShiftsMonitoring and ObservabilityML-Specific MetricsMonitoring ToolboxObservabilitySummary
Continual LearningStateless Retraining Versus Stateful TrainingWhy Continual Learning?Continual Learning ChallengesFour Stages of Continual LearningHow Often to Update Your ModelsTest in ProductionShadow DeploymentA/B TestingCanary ReleaseInterleaving ExperimentsBanditsSummary

Storage and ComputePublic Cloud Versus Private Data CentersDevelopment EnvironmentDev Environment SetupStandardizing Dev EnvironmentsFrom Dev to Prod: ContainersResource ManagementCron, Schedulers, and OrchestratorsData Science Workflow ManagementML PlatformModel DeploymentModel StoreFeature StoreBuild Versus BuySummary
User ExperienceEnsuring User Experience ConsistencyCombatting “Mostly Correct” PredictionsSmooth FailingTeam StructureCross-functional Teams CollaborationEnd-to-End Data ScientistsResponsible AIIrresponsible AI: Case StudiesA Framework for Responsible AISummary

Overview

Machine learning systems are both complex and unique. Complex because they consist of many different components and involve many different stakeholders. Unique because they're data dependent, with data varying wildly from one use case to the next. In this book, you'll learn a holistic approach to designing ML systems that are reliable, scalable, maintainable, and adaptive to changing environments and business requirements.

Author Chip Huyen, co-founder of Claypot AI, considers each design decision--such as how to process and create training data, which features to use, how often to retrain models, and what to monitor--in the context of how it can help your system as a whole achieve its objectives. The iterative framework in this book uses actual case studies backed by ample references.

This book will help you tackle scenarios such as:

Engineering data and choosing the right metrics to solve a business problem
Automating the process for continually developing, evaluating, deploying, and updating models
Developing a monitoring system to quickly detect and address issues your models might encounter in production
Architecting an ML platform that serves across use cases
Developing responsible ML systems