Chapter 1: Why Machine Learning Now

In this chapter, we review the history of machine learning and the limits of traditional approaches, and build an intuitive understanding of why machine learning is needed today. Through real-world success stories, we learn about the potential of machine learning.

Learning Objectives

By reading this chapter, you will be able to:

• Understand the historical evolution of machine learning (from the 1950s to the present)
• Explain the three limitations of traditional approaches
• Understand the social and technological context that makes machine learning necessary
• Learn the power of machine learning from 20 years of evolution in the Netflix recommendation system
• Grasp the overall picture of the machine learning pipeline

1.1 The History of Machine Learning: From Rules to Learning

Research into artificial intelligence (AI) and machine learning (ML) began at the dawn of computer science. Looking back over roughly 70 years of history, we can see a major paradigm shift from "teaching rules" to "learning from data."

The 1950s: The Dawn of AI and the Turing Test

1950 - Alan Turing, "Computing Machinery and Intelligence"

The British mathematician Alan Turing posed the question, "Can machines think?" The "Turing Test" he proposed is a test to judge whether a machine can return responses indistinguishable from those of a human. This became the starting point of AI research.

1956 - The Dartmouth Conference

The term "Artificial Intelligence" was officially born, coined by John McCarthy and others. At this conference, researchers made the optimistic declaration that "every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it."

The 1980s: The Rise and Limits of Expert Systems

The heyday of rule-based AI

In the 1980s, expert systems achieved commercial success. An expert system is a system that encodes the knowledge of human experts as "if-then" rules.

Success story: MYCIN (a medical diagnosis system)

• Development: Stanford University (1970s)
• Function: Diagnosis of bacterial infections and recommendation of antibiotics
• Number of rules: About 600 if-then rules
• Performance: Diagnostic accuracy on par with specialists (about 65%)

Limits of expert systems

1. Explosive growth in rules: Complex problems require thousands to tens of thousands of rules
2. Soaring maintenance costs: Every time new knowledge is added, the entire rule set must be reviewed
3. Difficulty handling exceptions: Cannot cope with unforeseen cases
4. Knowledge acquisition bottleneck: Extracting knowledge from experts is time-consuming

The 1990s: The Rise of Statistical Machine Learning

Paradigm shift: Instead of writing rules, learn from data

In the 1990s, researchers recognized the limits of expert systems and shifted to statistical machine learning, which "automatically discovers patterns from data."

Development of key methods

• Decision Tree: Automatically generates rules by branching the data
• Support Vector Machine (SVM): Excellent at classifying high-dimensional data
• Random Forest: Combines multiple decision trees to improve accuracy

Success story: Handwritten character recognition (MNIST)

Automatic postal-code reading systems were put into practical use, proving the effectiveness of statistical machine learning.

The 2000s: The Spread of Big Data and the Cloud

The age of the data explosion

In the 2000s, the spread of the Internet caused an explosive increase in the volume of data.

• 2000: World digital data volume - about 2 exabytes (2 × 10^18 bytes)
• 2010: About 2 zettabytes (2 × 10^21 bytes) - a 1,000-fold increase in 10 years
• 2020: About 59 zettabytes - a further 30-fold increase

The emergence of cloud computing

• 2006: Amazon Web Services (AWS) launched
• 2008: Google App Engine launched
• Impact: Anyone could now access large-scale computing resources

The 2010s: The Deep Learning Revolution

2012 - The ImageNet Revolution

In 2012, a team led by Geoffrey Hinton of the University of Toronto scored a decisive victory in the image recognition competition (ImageNet) with the deep learning model "AlexNet."

• Accuracy of traditional methods: Error rate of about 26%
• Accuracy of AlexNet: Error rate of about 16% - a 10% improvement
• Impact: A historic moment that triggered the deep learning boom

Characteristics of deep learning

1. Automatic feature extraction: No need for humans to design features
2. Hierarchical representation learning: Automatically builds high-level concepts from low-level features
3. Improved performance with large data: The more data, the higher the accuracy

The 2020s: Foundation Models and Generative AI

The impact of the Transformer architecture (from 2017)

The Transformer, proposed in the paper "Attention is All You Need" (2017) by a Google research team, revolutionized natural language processing.

• 2018: BERT (Google) - Greatly improved natural language understanding accuracy
• 2020: GPT-3 (OpenAI) - A giant language model with 175 billion parameters
• 2022: ChatGPT - A turning point for the spread of AI to general users
• 2023-2024: GPT-4, Claude, Gemini - Into the era of multimodal AI

The rise of Generative AI

• Text generation: ChatGPT, Claude, Bard
• Image generation: Stable Diffusion, Midjourney, DALL-E
• Video generation: Runway Gen-2, Pika
• Code generation: GitHub Copilot, Codex

Historical Timeline

1.2 The Limits of Traditional Approaches

Before machine learning appeared, software systems were designed mainly on a "rule-based" basis. However, this approach has serious limitations. Let's look at some concrete examples.

Challenge 1: The Problem of Rule Complexity and Maintainability

Concrete example: Spam email filter

In the 1990s, spam email filters operated on human-written rules.

if email_subject contains "Free":
    return "Spam"

if email_body contains "Click now":
    return "Spam"

if sender_address in blacklist:
    return "Spam"

# ... over 1,000 rules continue ...

Problems

1. Explosion in the number of rules: A practical filter requires over 1,000 rules
2. Soaring maintenance costs:
   • Adding rules every time a new spam technique appears
   • Checking for contradictions and conflicts between rules
   • Annual maintenance cost: 1 million to 5 million yen
3. A cat-and-mouse game with spammers:
   • "Free" → "Ｆｒｅｅ" (full-width) → "F.r.e.e" → "무료"
   • It is impossible to encode every variation as a rule

Solution via machine learning

Modern spam filters use machine learning:

• Training data: 100,000 emails (labeled as spam/normal)
• Learning result: The model automatically extracts the features of spam
• Accuracy: Over 99% (traditional rule-based methods are around 90%)
• Adaptability: Retraining on new data handles new techniques as well

Challenge 2: Lack of Scalability

Concrete example: Classifying product reviews

Suppose an e-commerce site wants to classify customer reviews as "positive" or "negative."

Traditional method (manual classification)

• Processing capacity: Number of reviews one worker can process per day - about 100
• Cost: 2,000 yen/hour × 8 hours = 16,000 yen/day
• The scaling problem:
  • At Amazon scale (1 million reviews per day), 10,000 workers would be required
  • Daily cost: 160 million yen
  • Annual cost: about 60 billion yen (realistically impossible)

Machine learning method

• Initial investment: Model development (1-2 weeks, cost around 1 million yen)
• Processing capacity: Over 1,000 items per second (when using a GPU server)
• Operating cost: Server costs of 100,000 to 500,000 yen per month
• Scalability: Even as the data volume grows, processing time increases sub-linearly

Challenge 3: Lack of Adaptability

Concrete example: Stock market trading strategies

In the financial industry, rule-based trading strategies known as "technical analysis" have been used for many years.

Traditional rule-based strategy

Rule 1: If the 25-day moving average crosses above the 75-day moving average, "Buy"
Rule 2: If the RSI (Relative Strength Index) is 30 or below, "Buy"; if 70 or above, "Sell"
Rule 3: If trading volume is 2x or more the 20-day average, "Watch"

Problems

1. Changes in the market environment:
   • Strategies that worked in the 2000s no longer work in the 2020s
   • The spread of algorithmic trading has changed market structure
   • Cannot respond to unpredictable events like COVID-19
2. Delays in updating rules:
   • It takes months to detect market changes
   • It takes months to develop and validate new rules
   • The market environment changes further before implementation is complete

Solution via machine learning

Modern hedge funds adopt machine-learning-based strategies:

• Adaptive learning: Retrained daily on new data
• Pattern discovery: Detects subtle correlations that humans miss
• Real-time response: Adapts to market changes in minutes to hours
• Results: ML-driven funds such as Renaissance Technologies achieve annual returns of 30-40%

Traditional Approaches vs. Machine Learning: A Comprehensive Comparison

1.3 Case Study: 20 Years of the Netflix Recommendation System

The Netflix recommendation system is the best example symbolizing the paradigm shift from rule-based to machine learning. Let's trace its 20-year evolution.

Phase 1: The Rule-Based Era (2000-2006)

The early approach: Simple rule matching

In its founding period (the DVD rental era), Netflix made recommendations using simple rules like the following:

if user rated "The Godfather" highly:
    recommendation = "Goodfellas", "Casino" (works by the same director)

if user watched 3 or more "sci-fi action" titles:
    recommendation = top 10 popular titles in the "sci-fi action" category

if user_age >= 40:
    recommendation = from the classic movies category

Performance metrics

• Accuracy: About 60% (probability that a recommended title is actually watched)
• Coverage: Only about 20% of all titles were recommended (skewed toward popular titles)
• Personalization: Low (same recommendations for users in the same category)

Problems

1. Lack of serendipity: No unexpected discoveries
2. Cold-start problem: Cannot make recommendations for new users
3. Scalability: Managing rules becomes difficult as the catalog grows

Phase 2: Introducing Machine Learning (2006-2012)

The Netflix Prize: A $1 million contest (2006-2009)

In 2006, Netflix held a contest offering $1 million in prize money to improve the accuracy of its recommendation system by 10%.

Contest details

• Dataset: 480,000 users, 18,000 titles, 100 million rating records
• Baseline: Netflix's existing algorithm (Cinematch)
• Goal: Improve RMSE by 10% (0.9525 → 0.8572)
• Participants: Over 50,000 people from 186 countries worldwide

Key method: Collaborative Filtering

Collaborative filtering is based on the hypothesis that "similar users like similar titles."

Basic idea

1. User A rated "Titanic," "The Notebook," and "Pretty Woman" highly
2. User B also rated the same 3 titles highly
3. User B also rated "La La Land" highly
4. → Recommend "La La Land" to User A

Technical evolution

• Matrix Factorization: Embeds users and titles into a low-dimensional space
• Ensemble learning: Combines multiple algorithms to improve accuracy
• Time-series patterns: Considers viewing time and day-of-week patterns

Results (2009)

• Accuracy: Improved to about 75% (a 15% improvement)
• Recommendation diversity: Long-tail titles also came to be recommended
• Business impact:
  • Customer satisfaction up 20%
  • Churn rate reduced by 10%
  • Estimated value: $1 billion per year

Phase 3: The Deep Learning Era (2012-Present)

2012: The shift to streaming

The transition from DVD rental to streaming caused an explosive increase in the available data.

New data sources

• Viewing behavior: Play, pause, fast-forward, rewind
• Watch time: How many minutes were watched (completion rate)
• Viewing device: TV, PC, smartphone, tablet
• Time of viewing: Weekday evening, weekend afternoon, etc.
• Visual information: Click-through rate of thumbnail images

The introduction of deep learning

Around 2015, Netflix began deploying deep neural networks in earnest.

Key technologies

1. Deep Neural Networks (DNN):
   • Learn complex patterns with millions of parameters
   • Model users' short-term and long-term preferences simultaneously
2. Convolutional Neural Networks (CNN):
   • Extract visual features from title poster images
   • Predict which thumbnail will attract the user
3. Recurrent Neural Networks (RNN):
   • Learn time-series patterns of viewing history
   • Predict "what the user will want to watch next"
4. Reinforcement Learning:
   • Optimize long-term user engagement
   • Emphasize completed viewing rather than mere click-through rate

Current performance (2024)

• Accuracy: About 85% (probability of watching a recommended title)
• Personalization:
  • Even for the same title, different thumbnails are shown to different users
  • Dozens of thumbnail variations for a single title
• Scale:
  • Over 230 million users
  • Over 500 million recommendations per day
  • 75% of watch time comes from recommendations

20 Years of Evolution: A Quantitative Comparison

Business Impact

Improved customer retention

• The recommendation system lets users quickly discover content they want to watch
• Result: churn rate reduced by about 30%
• 2024 churn rate: about 2% per month (industry average 5-7%)

Optimization of content investment

• Machine learning predicts "which content will become popular"
• Example: "House of Cards" was greenlit based on data analysis
• Result: an improved hit rate for original content

Accelerated global expansion

• Automatically generates recommendations suited to users in each country and region
• Expansion made possible in over 190 countries

1.4 Traditional Approaches vs. Machine Learning: A Workflow Comparison

Rule-based systems and machine learning systems differ fundamentally in their development and operational workflows. Let's compare them visually.

Workflow Comparison Diagram

Quantitative Comparison: Development and Operations

A Concrete Example on the Timeline

Scenario: A credit card fraud detection system

Traditional method timeline

• Month 1-2: Analyze past fraud patterns, design 100 rules
• Month 3-4: Code the rules, test
• Month 5: Deploy to production
• Month 6: A new fraud technique appears (accuracy drops)
• Month 7: Add new rules, retest
• Month 8: Redeploy
• → Repeat

Machine learning method timeline

• Week 1-2: Collect past transaction data (1 million records)
• Week 3-4: Feature design, model training
• Week 5-6: Evaluation and tuning
• Week 7: Deploy to production
• Week 8 onward: Automatic daily retraining on new data, automatic adaptation to new patterns

Results comparison

• Development period: 5 months → 7 weeks (about 70% shorter)
• Adaptation speed: 2 months → 1 day (about 60x faster)
• Accuracy: 75% → 95% (20% improvement)

1.5 Column: A Day in the Life of a Software Engineer

Let's look through concrete stories at how the daily work of engineers has changed with the advent of machine learning.

2005: An Engineer in the Rule-Based Era

A day in the life of Mr. Tanaka (age 28, e-commerce site development engineer)

9:00 - Morning standup and bug report review

Over the weekend, a bug occurred in the product recommendation system. The issue: single men who bought "kids' products" are endlessly recommended children's clothing.

9:30 - Investigating the cause

# The problematic code
if "kids" in purchase_history_categories:
    recommended_categories = ["kids", "baby", "toys"]
    # → Does not consider the possibility of a gift purchase!

10:30 - Fixing the rule

# Revised version
if "kids" in purchase_history_categories:
    if kids_product_purchase_count >= 3:
        # Multiple purchases likely mean a parent
        recommended_categories = ["kids", "baby", "toys"]
    else:
        # 1-2 purchases may be a gift
        recommended_categories = ["gifts", "general products"]

12:00 - Lunch break

13:00 - Testing

Run 100 test cases. 7 fail. A side effect of the rule broke other cases.

14:30 - Further fixes

Two hours spent adjusting priorities to resolve conflicts between rules.

16:30 - Review

A senior engineer reviews the fix. They point out, "This rule might conflict with the spring campaign rule."

17:30 - Re-fixing

Coordinating with the campaign rule.

19:00 - Finally deploying

Fixing a single bug took the entire day.

19:30 - A new bug report just before leaving...

"Men who bought products for pregnant women are endlessly recommended maternity products."

2025: An Engineer in the Machine Learning Era

A day in the life of Mr. Sato (age 30, ML engineer)

9:00 - Dashboard check

Check the state of the recommendation system on the monitoring dashboard:

• Click-through rate (CTR): 3.2% (within normal range)
• Conversion rate: 1.8% (within normal range)
• Model accuracy: 92.5% (target 90% or above)
• Anomaly detection: none

9:15 - A/B test result analysis

Compare the new model Ver2.0 released last week with Ver1.0:

• Ver2.0: CTR 3.2%, CV 1.8%
• Ver1.0: CTR 3.0%, CV 1.7%
• → Ver2.0 is statistically significantly better (p < 0.01)
• Decision: roll out Ver2.0 to all users

10:00 - Experimenting with a new feature

An experiment adding "the user's browsing time-of-day pattern" as a new feature:

# Adding features (Python code, 5 lines)
features['hour_preference'] = user_behavior.groupby('hour').mean()
features['weekend_preference'] = user_behavior[user_behavior['is_weekend']].mean()

# Retrain the model (run the automated pipeline)
model.fit(X_train, y_train)

# Evaluate
print(f"New model accuracy: {model.score(X_test, y_test):.2%}")
# Output: New model accuracy: 93.1%  (0.6% improvement!)

11:00 - Team meeting

Discussion with data scientists and the product manager:

• Next quarter's goals: CTR 3.5%, CV 2.0%
• New feature: developing a model that accounts for seasonality
• Computing resources: add 2 GPU servers to speed up training

13:00 - Lunch break

14:00 - Maintaining the model retraining pipeline

Add a monitoring rule to the retraining pipeline that runs automatically every day:

# Alert if accuracy drops by 5% or more
if model_accuracy < previous_accuracy * 0.95:
    send_alert("Model performance degradation detected!")

15:00 - Writing a technical blog post

Summarize this week's experiment results on the internal technical blog. Knowledge sharing.

16:00 - Online study group

Read the latest paper "Attention Mechanisms in Recommender Systems" together with colleagues.

17:30 - Leaving work

Tomorrow morning, just checking the results of the retraining that ran automatically overnight is enough.

2005 vs. 2025: A Comparison of Engineers' Work

Key point

Machine learning transformed engineers from "rule maintenance workers" into "problem solvers who use data and models." Much of the routine work has been automated, allowing humans to focus on more creative and strategic work.

1.6 Why Machine Learning "Now": Three Tailwinds

The concept of machine learning itself has existed since the 1950s, but its practical application began in earnest only from the 2010s. Why "now"? There are three major tailwinds.

Tailwind 1: A Dramatic Increase in Computing Power

Moore's Law and the exponential growth of computing power

In 1965, Intel co-founder Gordon Moore predicted that "the number of transistors in an integrated circuit doubles every two years." This law held for over 50 years, and computing power improved dramatically.

• 1990: Weeks to train a single neural network
• 2000: Days
• 2010: Hours
• 2020: Minutes to tens of minutes

The GPU (Graphics Processing Unit) revolution

The 2012 success of AlexNet was due to parallel computation using NVIDIA GPUs.

The democratization of cloud computing

With the arrival of AWS in 2006, anyone could access supercomputer-class computing resources.

• AWS EC2 GPU instance (p3.2xlarge):
  • NVIDIA V100 GPU × 1
  • About $3 per hour (about 400 yen)
  • About 10,000 yen even if used all day
• Google Colab Pro:
  • Access to a high-performance GPU for $10/month (about 1,300 yen)
  • Even students and individual researchers can conduct cutting-edge research

The dramatic drop in computing costs

• 1990: About $10,000 per GFLOPS
• 2000: About $100 per GFLOPS
• 2010: About $1 per GFLOPS
• 2020: About $0.01 per GFLOPS (a one-millionth reduction)

Tailwind 2: The Explosive Growth of Data

The exponential growth of digital data

• 2000: World data volume - about 2 exabytes
• 2010: About 2 zettabytes (1,000×)
• 2020: About 59 zettabytes (30×)
• 2025 (projected): About 175 zettabytes (3×)

The spread of IoT (Internet of Things)

• 2015: About 15 billion connected devices
• 2020: About 30 billion
• 2025 (projected): About 75 billion
• Data generation sources: Smartphones, wearables, self-driving cars, factory sensors, etc.

Lower-cost data collection

• Storage cost:
  • 2000: about $10 per GB
  • 2024: about $0.02 per GB (1/500)
• Impact: "Store rather than discard" data has become the norm

Major data sources

1. Social media:
   • Facebook: generates 4 petabytes of data per day
   • YouTube: 500 hours of video uploaded every minute
2. E-commerce:
   • Amazon: hundreds of millions of transaction logs per day
3. Sensor data:
   • Self-driving cars: a single vehicle generates 4 terabytes of data per day

Tailwind 3: The Evolution of Algorithms and Social Urgency

The emergence of breakthrough technologies

• 2012: AlexNet
  • Proved the effectiveness of Convolutional Neural Networks (CNN)
  • Improved image recognition accuracy by 10%
• 2014: Generative Adversarial Networks (GAN)
  • Enabled the generation of realistic images and video
• 2017: Transformer
  • Revolutionized natural language processing
  • The foundational technology of GPT, BERT, and ChatGPT
• 2020s: Foundation Models
  • A single model handles diverse tasks
  • GPT-4, Claude, Gemini, and others

The influence of the U.S. Materials Genome Initiative (MGI)

In 2011, the Obama administration launched the MGI to accelerate materials development, and the same data-driven approach spread to other fields as well.

Rising social urgency

1. Responding to climate change:
   • The 2015 Paris Agreement: hold warming to within 2°C
   • Renewable energy, energy storage, and CO₂-reduction technologies are urgently needed
   • Machine learning enables efficient development of materials and systems
2. Innovation in medicine and drug discovery:
   • AI played a role in COVID-19 vaccine development (mRNA vaccine design)
   • New drug development period: traditionally 10-15 years → shortened to 5-7 years with AI (target)
3. Intensifying international competition:
   • China: AI investment of trillions of yen per year as a national strategy
   • United States: competition over technological supremacy, such as the CHIPS Act (semiconductor support)
   • Japan: accelerating AI talent development under AI Strategy 2023

Summary: Three tailwinds are blowing simultaneously

Machine learning is a technology that is needed precisely "now," when technological maturity and social necessity have been satisfied at the same time.

1.7 The Standard Machine Learning Pipeline

So far, we have seen the necessity and potential of machine learning. So how does a machine learning project actually proceed? Let's look at the standard pipeline.

The 7 Steps of a Machine Learning Project

Details of Each Step

Step 0: Problem Formulation

Purpose: Define the problem you want to solve as a machine learning task

Key questions:

• What do you want to predict? (target variable)
• What data can you use? (input variables)
• What is the criterion for success? (evaluation metric)
• What are the constraints? (time, cost, accuracy)

Example:

• Task: Classify whether an email is spam
• Goal: Accuracy of 95% or more, false-positive rate of 1% or less
• Data: 100,000 labeled emails
• Constraint: Real-time judgment (within 100 ms)

Time required: 1-2 weeks

Step 1: Data Collection

Purpose: Gather the data to be used for training, validation, and testing

Data sources:

• Internal databases
• Public datasets (Kaggle, UCI ML Repository)
• Retrieval from APIs (Twitter API, Google Trends, etc.)
• Web scraping
• Sensor and log data

Data splitting:

• Training data: 70% (for model learning)
• Validation data: 15% (for hyperparameter tuning)
• Test data: 15% (for final evaluation)

Time required: 1 week to 1 month

Step 2: Data Preprocessing

Purpose: Clean and transform data into a format the model can handle

Key tasks:

1. Handling missing values: Deletion, mean imputation, predictive imputation
2. Outlier removal: Detecting and removing anomalous data
3. Data type conversion: Encoding categorical variables
4. Normalization/standardization: Unifying scales
5. Deduplication: Removing identical data

Time required: A few days to 2 weeks (depends on data quality)

Step 3: Feature Engineering

Purpose: Design features that are easy for the model to learn from

Methods:

• Feature selection: Selecting only the important features
• Feature generation: Creating new features from existing ones
• Dimensionality reduction: Reducing dimensions with PCA (Principal Component Analysis), etc.

Example (spam filter):

• Number of words in the email body
• Proportion of uppercase letters
• Frequency of keywords such as "free" and "click"
• Number of URLs
• Sender domain

Time required: 1-3 weeks (a crucial step)

Step 4: Model Training

Purpose: Learn patterns from the data

Key algorithms:

• Linear models: Linear regression, logistic regression
• Decision-tree-based: Random Forest, XGBoost, LightGBM
• Neural networks: Multi-layer Perceptron, CNN, RNN
• Support Vector Machine (SVM)

Training process:

1. Optimize the model's parameters on the training data
2. Minimize the loss function
3. Check performance on the validation data

Time required: A few hours to a few days (depends on data volume and model complexity)

Step 5: Model Evaluation

Purpose: Objectively evaluate the model's performance

Key evaluation metrics:

• Classification tasks: Accuracy, precision, recall, F1 score, AUC
• Regression tasks: MAE (Mean Absolute Error), RMSE (Root Mean Squared Error), R²

Evaluation methods:

• Performance evaluation on test data
• Cross-Validation
• Analysis of the Confusion Matrix

Time required: 1-3 days

Step 6: Deployment

Purpose: Deploy the trained model to a production environment

Deployment methods:

• REST API: Build an API server with Flask or FastAPI
• Cloud services: AWS SageMaker, Google Vertex AI
• Edge devices: TensorFlow Lite, ONNX Runtime

Time required: 1-2 weeks

Step 7: Monitoring & Maintenance

Purpose: Detect degradation in model performance and improve continuously

Monitoring items:

• Trend in prediction accuracy
• Data drift (changes in the input data distribution)
• Latency (response time)
• Error rate

Maintenance work:

• Periodic retraining (weekly, monthly)
• Adding new data
• Model version management

Ongoing work

Key Points of the Standard Pipeline

1. An iterative process: You cannot build a perfect model in one pass. Repeat experiment → evaluate → improve
2. Data quality matters most: "Garbage in, garbage out." You cannot build a good model from poor-quality data
3. Problem formulation is key: If Step 0 is inadequate, all subsequent steps are wasted
4. Feature engineering makes the difference: In many cases, feature design affects accuracy more than the choice of algorithm
5. The real work begins after deployment: Training is not the end; continuous monitoring and improvement are needed

We will study each step of this pipeline in detail from Chapter 2 onward. Let's deepen our understanding hands-on, using actual Python code.

Chapter Summary

What We Learned

1. The history of machine learning
   • 1950s: The dawn of AI (the Turing Test)
   • 1980s: The rise and limits of expert systems
   • 1990s: The rise of statistical machine learning
   • 2012: The deep learning revolution (AlexNet)
   • 2020s: Foundation models and generative AI
   • Over 70 years, an evolution from "teaching rules" to "learning from data"
2. The three limits of traditional approaches
   • Rule complexity: Over 1,000 rules for a spam filter, annual maintenance cost of 1-5 million yen
   • Lack of scalability: Manual classification is limited to 100 items/day, while machine learning enables 1 million items/day
   • Lack of adaptability: Responding to market changes takes months, while machine learning takes hours to days
3. 20 years of evolution in the Netflix recommendation system
   • Phase 1 (2000-2006): Rule-based, 60% accuracy
   • Phase 2 (2006-2012): Collaborative filtering, 75% accuracy, $1 billion/year in value
   • Phase 3 (2012-present): Deep learning, 85% accuracy, 75% of watch time via recommendations
4. Workflow comparison
   • Traditional method: a loop of rule design → implementation → testing → tuning (slow)
   • Machine learning: data collection → training → prediction → retraining (fast, adaptive)
   • Development time reduced by 60-80%, costs reduced by 90-99%
5. The change in engineers' work
   • 2005: Fixing rules all day (80% routine work)
   • 2025: Model design and experimentation (60% creative work)
   • Automation allows a focus on more strategic work
6. Three reasons machine learning is needed "now"
   • Computing power: GPU revolution, spread of the cloud, cost of 1/1,000,000
   • Data: 175 zettabytes in 2025, 75 billion IoT devices
   • Algorithms + social urgency: Transformer, foundation models, climate change, medicine, international competition
7. The 7 steps of the machine learning pipeline
   • Problem formulation → data collection → preprocessing → feature engineering → training → evaluation → deployment → monitoring
   • An iterative process; data quality matters most

Key Points

• Machine learning is not "magic," but a technology fusing data, computation, and algorithms
• Compared with traditional approaches, it holds an overwhelming advantage in speed, scale, and adaptability
• "Now"—when the three tailwinds (computation, data, and social urgency) are blowing at once—is the optimal timing
• Machine learning does not replace humans; it enables more creative work

On to the Next Chapter

In Chapter 2, we will study the fundamental theory of machine learning:

• Supervised learning vs. unsupervised learning vs. reinforcement learning
• Classification vs. regression
• The roles of training, validation, and test data
• Overfitting and generalization performance
• The bias-variance tradeoff

Furthermore, we will implement simple machine learning using Python. Get ready, and let's move on to the next chapter!

Exercises

Exercise 1 (Difficulty: easy)

In the history of machine learning, what was the 2012 event that triggered attention toward deep learning? Also, give the name of the model and the results it achieved.

Exercise 2 (Difficulty: easy)

List the three main limitations of traditional rule-based approaches and briefly explain each.

Exercise 3 (Difficulty: medium)

Summarize how the Netflix recommendation system evolved from Phase 1 (rule-based, 2000-2006) to Phase 3 (deep learning, 2012-present) from the following perspectives:

1. Technical evolution (methods used)
2. Improvement in accuracy
3. Business impact

References

1. Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.
   URL: https://www.deeplearningbook.org/
2. Russell, S., & Norvig, P. (2020). Artificial Intelligence: A Modern Approach (4th ed.). Pearson.
   ISBN: 978-0134610993
3. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). "ImageNet Classification with Deep Convolutional Neural Networks." Advances in Neural Information Processing Systems, 25.
   URL: NeurIPS 2012
4. Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). "Attention is All You Need." Advances in Neural Information Processing Systems, 30.
   DOI: arXiv:1706.03762
5. Gomez-Uribe, C. A., & Hunt, N. (2015). "The Netflix Recommender System: Algorithms, Business Value, and Innovation." ACM Transactions on Management Information Systems, 6(4), 1-19.
   DOI: 10.1145/2843948
6. LeCun, Y., Bengio, Y., & Hinton, G. (2015). "Deep learning." Nature, 521(7553), 436-444.
   DOI: 10.1038/nature14539
7. Jordan, M. I., & Mitchell, T. M. (2015). "Machine learning: Trends, perspectives, and prospects." Science, 349(6245), 255-260.
   DOI: 10.1126/science.aaa8415