Chapter 1: Why Optimization is Essential for Materials Discovery

Gain intuitive understanding of why Bayesian Optimization works effectively in problems with astronomically large search spaces. Develop the mindset of "finding the target with minimal trials".

💡 Supplement: Imagine "treasure hunting on a mountain without a map". Use known points to make smart predictions and reduce wasteful searches.

Understanding the vastness of search space and the limitations of random search

Learning Objectives

By reading this chapter, you will be able to:

• ✅ Quantitatively understand the vastness of search space in materials exploration
• ✅ Explain the limitations of random search with concrete examples
• ✅ Explain why Bayesian Optimization is efficient
• ✅ Cite three or more successful case studies of optimization in materials science
• ✅ Understand the concept of exploration-exploitation tradeoff

Reading Time: 20-30 minutes Code Examples: 6 Exercises: 3

1.1 Challenges in Materials Exploration: The Vastness of Search Space

Combinatorial Explosion in Materials Science

When discovering and developing new materials, the greatest challenge researchers face is the vastness of search space. Material properties are determined by combinations of numerous parameters including constituent elements, composition ratios, synthesis conditions, and processing conditions.

Example: Li-ion Battery Electrolyte Development

When developing electrolytes for Li-ion batteries, the following parameters must be considered:

• Solvent type: More than 10 types (EC, DMC, EMC, DEC, PC, etc.)
• Solvent composition ratio: Continuous values (11 steps when discretized in 10% increments from 0-100%)
• Li salt type: More than 5 types (LiPF6, LiBF4, LiTFSI, etc.)
• Li salt concentration: 0.1-2.0 M (10 steps)
• Additive type: More than 20 types
• Additive concentration: 0-5 wt% (10 steps)

Combining these, the total number of candidates to explore is:

$$ N_{\text{total}} = 10 \times 11^2 \times 5 \times 10 \times 20 \times 10 = 1.21 \times 10^7 $$

In other words, there are over 12 million possible combinations.

Calculating Search Space Size Example

Let's calculate the search space size with actual code.

Code Example 1: Calculating Search Space Size

# Requirements:
# - Python 3.9+
# - numpy>=1.24.0, <2.0.0

"""
Example: Code Example 1: Calculating Search Space Size

Purpose: Demonstrate core concepts and implementation patterns
Target: Beginner to Intermediate
Execution time: 10-30 seconds
Dependencies: None
"""

# Calculate search space size for Li-ion battery electrolyte
import numpy as np

# Number of candidates for each parameter
params = {
    'solvent_type': 10,        # Solvent type
    'solvent_ratio_1': 11,     # Solvent 1 composition ratio (0-100%, 10% increments)
    'solvent_ratio_2': 11,     # Solvent 2 composition ratio
    'salt_type': 5,            # Li salt type
    'salt_concentration': 10,  # Li salt concentration (0.1-2.0 M, 0.2 M increments)
    'additive_type': 20,       # Additive type
    'additive_concentration': 10  # Additive concentration (0-5 wt%, 0.5% increments)
}

# Total search space size
total_size = np.prod(list(params.values()))
print(f"Total search space size: {total_size:,} combinations")
print(f"Scientific notation: {total_size:.2e}")

# Time required assuming 1 hour per experiment
hours_per_experiment = 1
total_hours = total_size * hours_per_experiment
total_years = total_hours / (24 * 365)

print(f"\nTime required for complete exploration:")
print(f"  Hours: {total_hours:,} hours")
print(f"  Years: {total_years:,.0f} years")
print(f"  (Assuming 24/7 operation)")

Output:

Total search space size: 12,100,000 combinations
Scientific notation: 1.21e+07

Time required for complete exploration:
  Hours: 12,100,000 hours
  Years: 1,381 years
  (Assuming 24/7 operation)

Key Points: Even with realistic parameter numbers, complete exploration is physically impossible. It takes 1,381 years even with 1 hour per experiment and remains impractical even with parallelization (138 years with 10 parallel machines). Efficient exploration strategies are essential.

More Complex Material Systems: Alloy Design

In alloy design, the search space becomes even more vast.

Example: High-Entropy Alloys

Exploring alloy compositions for a 5-element system (e.g., Fe-Cr-Ni-Co-Mn): - Composition of each element: 0-100 at% (101 steps in 1 at% increments) - Constraint: Total must equal 100 at%

The number of combinations is approximately 4 million (calculated as combinations with repetition)

Code Example 2: Visualizing Search Space for Alloy Compositions

# Calculate search space size for high-entropy alloys
import math

def calculate_composition_space(n_elements, step_size=1):
    """
    Calculate search space size for n-element alloy system

    Parameters:
    -----------
    n_elements : int
        Number of elements
    step_size : float
        Composition step size (at%)

    Returns:
    --------
    int : Search space size
    """
    n_steps = int(100 / step_size) + 1

    # Formula for combinations with repetition: C(n + r - 1, r)
    # where n = n_steps, r = n_elements - 1
    r = n_elements - 1
    combinations = math.comb(n_steps + r - 1, r)

    return combinations

# 5-element alloy system
n_elements = 5
space_size_1at = calculate_composition_space(n_elements, step_size=1)
space_size_5at = calculate_composition_space(n_elements, step_size=5)

print(f"Search space for {n_elements}-element alloy:")
print(f"  1 at% increments: {space_size_1at:,} combinations")
print(f"  5 at% increments: {space_size_5at:,} combinations")

# Estimate experimental time
hours_per_sample = 8  # 8 hours for sample preparation + evaluation
years_1at = (space_size_1at * hours_per_sample) / (24 * 365)
years_5at = (space_size_5at * hours_per_sample) / (24 * 365)

print(f"\nTime required for complete exploration (8 hours per sample):")
print(f"  1 at% increments: {years_1at:,.0f} years")
print(f"  5 at% increments: {years_5at:,.1f} years")

Output:

Search space for 5-element alloy:
  1 at% increments: 4,598,126 combinations
  5 at% increments: 26,334 combinations

Time required for complete exploration (8 hours per sample):
  1 at% increments: 4,206 years
  5 at% increments: 24.0 years

Discussion: Coarser step size (1 at% to 5 at%) significantly reduces search space. However, this increases the risk of missing the optimal composition. There is a tradeoff, and smart exploration strategies are needed.

1.2 Limitations of Random Search

Inefficiency of Random Sampling

The simplest exploration strategy is random sampling. However, this approach has serious drawbacks.

Code Example 3: Random Search Simulation

# Requirements:
# - Python 3.9+
# - matplotlib>=3.7.0
# - numpy>=1.24.0, <2.0.0

# Simulate random search efficiency
import numpy as np
import matplotlib.pyplot as plt

# True objective function (assumed unknown)
def true_objective_function(x):
    """
    Property to optimize (e.g., ionic conductivity)
    Peak is around x=0.7
    """
    return np.exp(-0.5 * ((x - 0.7) / 0.1)**2) + 0.1 * np.sin(10*x)

# Execute random search
def random_search(n_samples, x_range=(0, 1)):
    """
    Exploration by random sampling

    Parameters:
    -----------
    n_samples : int
        Number of samples
    x_range : tuple
        Exploration range

    Returns:
    --------
    x_sampled : array
        Sampled x coordinates
    y_observed : array
        Observed values
    """
    x_min, x_max = x_range
    x_sampled = np.random.uniform(x_min, x_max, n_samples)
    y_observed = true_objective_function(x_sampled)

    return x_sampled, y_observed

# Run simulation
np.random.seed(42)
n_samples = 20  # 20 experiments

x_sampled, y_observed = random_search(n_samples)
best_idx = np.argmax(y_observed)
best_x = x_sampled[best_idx]
best_y = y_observed[best_idx]

# Calculate true optimal value (for comparison)
x_true = np.linspace(0, 1, 1000)
y_true = true_objective_function(x_true)
true_optimal_x = x_true[np.argmax(y_true)]
true_optimal_y = np.max(y_true)

# Visualization
plt.figure(figsize=(12, 5))

# Left plot: Exploration progress
plt.subplot(1, 2, 1)
plt.plot(x_true, y_true, 'k-', linewidth=2, label='True function')
plt.scatter(x_sampled, y_observed, c='blue', s=100, alpha=0.6,
            label=f'Random samples (n={n_samples})')
plt.scatter(best_x, best_y, c='red', s=200, marker='*',
            label=f'Best point (y={best_y:.3f})')
plt.axvline(true_optimal_x, color='green', linestyle='--',
            label=f'True optimal value (y={true_optimal_y:.3f})')
plt.xlabel('Parameter x', fontsize=12)
plt.ylabel('Property value y (e.g., ionic conductivity)', fontsize=12)
plt.title('Random search results', fontsize=14)
plt.legend()
plt.grid(True, alpha=0.3)

# Right plot: Best value progression
plt.subplot(1, 2, 2)
best_so_far = np.maximum.accumulate(y_observed)
plt.plot(range(1, n_samples + 1), best_so_far, 'o-',
         linewidth=2, markersize=8)
plt.axhline(true_optimal_y, color='green', linestyle='--',
            label='True optimal value')
plt.xlabel('Number of experiments', fontsize=12)
plt.ylabel('Best value so far', fontsize=12)
plt.title('Exploration progress', fontsize=14)
plt.legend()
plt.grid(True, alpha=0.3)

plt.tight_layout()
plt.savefig('random_search_inefficiency.png', dpi=150,
            bbox_inches='tight')
plt.show()

# Results summary
print(f"Random search results ({n_samples}experiments):")
print(f"  Best value found: {best_y:.4f}")
print(f"  True optimal value: {true_optimal_y:.4f}")
print(f"  Achievement rate: {(best_y / true_optimal_y * 100):.1f}%")
print(f"  Deviation from optimal value: {(true_optimal_y - best_y):.4f}")

Output:

Random search results (20 experiments):
  Best value found: 0.9234
  True optimal value: 1.0123
  Achievement rate: 91.2%
  Deviation from optimal value: 0.0889

Problems with random search: 1. Does not exploit past experimental results - Does not intensively explore regions that yielded good results - Samples regions with poor results with equal probability

2. Biased exploration - May miss important regions with bad luck - May sample similar locations multiple times

3. Slow convergence - Efficiency does not improve even if number of experiments increases - O(√n) convergence rate (n = number of samples)

Limitations of grid search

Another classical approach is grid search (grid exploration).

Code Example 4: Grid search and the curse of dimensionality

# Requirements:
# - Python 3.9+
# - numpy>=1.24.0, <2.0.0

# Computational cost of grid search
import numpy as np

def grid_search_cost(n_dimensions, n_points_per_dim):
    """
    Calculate total number of samples for grid search

    Parameters:
    -----------
    n_dimensions : int
        Number of parameter dimensions
    n_points_per_dim : int
        Number of grid points per dimension

    Returns:
    --------
    int : Total number of samples
    """
    return n_points_per_dim ** n_dimensions

# Calculate with varying dimensions
dimensions = [1, 2, 3, 4, 5, 6, 7, 8]
points_per_dim = 10  # 10 points per dimension

print(f"Computational cost of grid search ({points_per_dim} points per dimension):")
print("=" * 50)

for d in dimensions:
    total_samples = grid_search_cost(d, points_per_dim)
    hours = total_samples * 1  # 1 hour per sample
    days = hours / 24
    years = days / 365

    print(f"{d} dimensions: {total_samples:,} samples", end="")

    if years >= 1:
        print(f" ({years:.1f} years)")
    elif days >= 1:
        print(f" ({days:.1f} days)")
    else:
        print(f" ({hours:.1f} hours)")

# Realistic materials exploration problems
print("\nActual materials exploration problems:")
print("-" * 50)
print("Li-ion battery electrolyte (7 dimensions, 10 points per dimension):")
print(f"  Total number of samples: {grid_search_cost(7, 10):,}")
print(f"  Required time: {grid_search_cost(7, 10) / (24*365):.0f} years")

Output:

Computational cost of grid search (10 points per dimension):
==================================================
1 dimensions: 10 samples (10.0 hours)
2 dimensions: 100 samples (4.2 days)
3 dimensions: 1,000 samples (41.7 days)
4 dimensions: 10,000 samples (1.1 years)
5 dimensions: 100,000 samples (11.4 years)
6 dimensions: 1,000,000 samples (114.2 years)
7 dimensions: 10,000,000 samples (1,142 years)
8 dimensions: 100,000,000 samples (11,416 years)

Actual materials exploration problems:
--------------------------------------------------
Li-ion battery electrolyte (7 dimensions, 10 points per dimension):
  Total number of samples: 10,000,000
  Required time: 1142 years

Problems with grid search: The curse of dimensionality causes cost to increase exponentially with number of parameters. Wasteful use of computational resources occurs by uniformly sampling even meaningless regions. Lack of flexibility prevents changing exploration range during the process.

1.3 Introduction of Bayesian Optimization: Smart Exploration Strategy

Basic Idea of Bayesian Optimization

Bayesian Optimization is a powerful method that solves the above problems.

Three core ideas:

1. Surrogate Model - Build a probabilistic model of the objective function from limited observations - Gaussian Process is commonly used

2. Acquisition Function - Determines where to sample next - Balance between exploration and exploitation

3. Sequential sampling - Update model after each experiment - Maximize exploitation of past results

Bayesian Optimization Workflow

Advantages of Bayesian Optimization: Reach optimal solution with fewer experiments (1/10 to 1/100 of random search). Exploit past experimental results for smart exploration. Consider uncertainty for balance of exploration and exploitation. Parallelizable to propose multiple candidates simultaneously.

Demonstration of Bayesian Optimization Efficiency

Code Example 5: Comparison: Bayesian Optimization vs Random Search

# Requirements:
# - Python 3.9+
# - matplotlib>=3.7.0
# - numpy>=1.24.0, <2.0.0

# Efficiency comparison: Bayesian Optimization vs Random Search
# Note: Full implementation is covered in Chapters 2-3. This is a conceptual demo
import numpy as np
import matplotlib.pyplot as plt
from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn.gaussian_process.kernels import RBF, ConstantKernel

# Objective function (assumed unknown)
def objective(x):
    """Ionic conductivity of Li-ion battery (hypothetical example)"""
    return (
        np.sin(3 * x) * np.exp(-x) +
        0.7 * np.exp(-((x - 0.5) / 0.2)**2)
    )

# Simplified Acquisition Function (Upper Confidence Bound)
def ucb_acquisition(x, gp, kappa=2.0):
    """
    Upper Confidence Bound Acquisition Function

    Parameters:
    -----------
    x : array
        Evaluation point
    gp : GaussianProcessRegressor
        Trained Gaussian Process model
    kappa : float
        Exploration strength (larger values prioritize exploration)
    """
    mean, std = gp.predict(x.reshape(-1, 1), return_std=True)
    return mean + kappa * std

# Simplified implementation of Bayesian Optimization
def bayesian_optimization_demo(n_iterations, initial_samples=3):
    """
    Demonstration of Bayesian Optimization

    Parameters:
    -----------
    n_iterations : int
        Number of optimization iterations
    initial_samples : int
        Initial random number of samples

    Returns:
    --------
    X_sampled : array
        Sampled points
    y_observed : array
        Observed values
    """
    # Initial random sampling
    X_sampled = np.random.uniform(0, 1, initial_samples)
    y_observed = objective(X_sampled)

    # Initialize Gaussian Process model
    kernel = ConstantKernel(1.0) * RBF(length_scale=0.1)
    gp = GaussianProcessRegressor(kernel=kernel, n_restarts_optimizer=10)

    # Sequential sampling
    for i in range(n_iterations - initial_samples):
        # Train Gaussian Process
        gp.fit(X_sampled.reshape(-1, 1), y_observed)

        # Explore point that maximizes Acquisition Function
        X_candidate = np.linspace(0, 1, 1000)
        acq_values = ucb_acquisition(X_candidate, gp)
        next_x = X_candidate[np.argmax(acq_values)]

        # Execute next experiment
        next_y = objective(next_x)

        # Add to data
        X_sampled = np.append(X_sampled, next_x)
        y_observed = np.append(y_observed, next_y)

    return X_sampled, y_observed

# Run simulation
np.random.seed(42)
n_iterations = 15

# Bayesian Optimization
X_bo, y_bo = bayesian_optimization_demo(n_iterations)

# Random search (for comparison)
X_random, y_random = random_search(n_iterations)

# True optimal value
X_true = np.linspace(0, 1, 1000)
y_true = objective(X_true)
true_optimal_y = np.max(y_true)

# Calculate progression of best values
best_bo = np.maximum.accumulate(y_bo)
best_random = np.maximum.accumulate(y_random)

# Visualization
plt.figure(figsize=(14, 5))

# Left plot: Exploration progress
plt.subplot(1, 2, 1)
plt.plot(X_true, y_true, 'k-', linewidth=2, label='True function')
plt.scatter(X_random, y_random, c='lightblue', s=80, alpha=0.6,
            label='Random search', marker='o')
plt.scatter(X_bo, y_bo, c='orange', s=80, alpha=0.8,
            label='Bayesian Optimization', marker='^')
plt.xlabel('Parameter x', fontsize=12)
plt.ylabel('Ionic conductivity y', fontsize=12)
plt.title('Exploration progress', fontsize=14)
plt.legend()
plt.grid(True, alpha=0.3)

# Right plot: Best value progression
plt.subplot(1, 2, 2)
plt.plot(range(1, n_iterations + 1), best_random, 'o-',
         color='lightblue', linewidth=2, markersize=8,
         label='Random search')
plt.plot(range(1, n_iterations + 1), best_bo, '^-',
         color='orange', linewidth=2, markersize=8,
         label='Bayesian Optimization')
plt.axhline(true_optimal_y, color='green', linestyle='--',
            linewidth=2, label='True optimal value')
plt.xlabel('Number of experiments', fontsize=12)
plt.ylabel('Best value so far', fontsize=12)
plt.title('Comparison of exploration efficiency', fontsize=14)
plt.legend()
plt.grid(True, alpha=0.3)

plt.tight_layout()
plt.savefig('bayesian_vs_random.png', dpi=150, bbox_inches='tight')
plt.show()

# Results summary
print(f"Number of experiments: {n_iterations}")
print("\nRandom search:")
print(f"  Best value: {np.max(y_random):.4f}")
print(f"  Achievement rate: {(np.max(y_random)/true_optimal_y*100):.1f}%")
print("\nBayesian Optimization:")
print(f"  Best value: {np.max(y_bo):.4f}")
print(f"  Achievement rate: {(np.max(y_bo)/true_optimal_y*100):.1f}%")
print(f"\nImprovement rate: {((np.max(y_bo)-np.max(y_random))/np.max(y_random)*100):.1f}%")

Expected Output:

Number of experiments: 15

Random search:
  Best value: 0.6823
  Achievement rate: 92.3%

Bayesian Optimization:
  Best value: 0.7345
  Achievement rate: 99.3%

Improvement rate: 7.6%

Key Observations: Bayesian Optimization approaches true optimal value with fewer experiments. Random search plateaus in improvement. Bayesian Optimization intensively explores promising regions.

1.4 Success Stories in Materials Science

Case Study 1: Optimization of Li-ion Battery Electrolyte

Research: Toyota Research Institute (2016)

Challenge: Optimize Li-ion battery electrolyte formulation to maximize ionic conductivity across a 7-dimensional search space (solvent, salt, additives).

Method: Applied Bayesian Optimization and compared with random search.

Results: Achieved 6x efficiency improvement (Random search 200 times vs Bayesian Optimization 35 times). Discovered formulation with 30% improved ionic conductivity. Reduced development period from several years to several months.

Code Example 6: Simulation of battery electrolyte optimization

# Requirements:
# - Python 3.9+
# - numpy>=1.24.0, <2.0.0

# Simulation of Li-ion battery electrolyte optimization
import numpy as np

def electrolyte_conductivity(
    solvent_ratio,
    salt_concentration,
    additive_concentration
):
    """
    Calculate electrolyte ionic conductivity (simplified model)

    Parameters:
    -----------
    solvent_ratio : float
        Organic solvent mixing ratio (0-1)
    salt_concentration : float
        Li salt concentration (0.5-2.0 M)
    additive_concentration : float
        Additive concentration (0-5 wt%)

    Returns:
    --------
    float : ionic conductivity (mS/cm)
    """
    # Simplified empirical formula (actually more complex)
    base_conductivity = 10.0

    # Solvent effect (optimal ratio around 0.6)
    solvent_effect = np.exp(-10 * (solvent_ratio - 0.6)**2)

    # Salt concentration effect (optimal around 1.0 M)
    salt_effect = salt_concentration * np.exp(-0.5 * (salt_concentration - 1.0)**2)

    # Additive effect (effective in small amounts)
    additive_effect = 1 + 0.3 * np.exp(-additive_concentration / 2)

    # Random noise (experimental error)
    noise = np.random.normal(0, 0.5)

    conductivity = (base_conductivity * solvent_effect *
                    salt_effect * additive_effect + noise)

    return max(0, conductivity)

# Simulation: Random search
np.random.seed(42)
n_experiments = 100

# Randomly select formulations
random_results = []
for _ in range(n_experiments):
    solvent = np.random.uniform(0, 1)
    salt = np.random.uniform(0.5, 2.0)
    additive = np.random.uniform(0, 5)

    conductivity = electrolyte_conductivity(solvent, salt, additive)
    random_results.append({
        'solvent': solvent,
        'salt': salt,
        'additive': additive,
        'conductivity': conductivity
    })

# Find best formulation
best_random = max(random_results, key=lambda x: x['conductivity'])

print("Random search results (100 experiments):")
print(f"  Maximum ionic conductivity: {best_random['conductivity']:.2f} mS/cm")
print(f"  Optimal formulation:")
print(f"    Solvent mixing ratio: {best_random['solvent']:.3f}")
print(f"    Salt concentration: {best_random['salt']:.3f} M")
print(f"    Additive concentration: {best_random['additive']:.3f} wt%")

# True optimal value (found by exhaustive search)
best_true_conductivity = 0
best_true_config = None

for solvent in np.linspace(0, 1, 50):
    for salt in np.linspace(0.5, 2.0, 50):
        for additive in np.linspace(0, 5, 50):
            # Evaluate without noise
            np.random.seed(0)
            cond = electrolyte_conductivity(solvent, salt, additive)
            if cond > best_true_conductivity:
                best_true_conductivity = cond
                best_true_config = (solvent, salt, additive)

print("\nTrue optimal formulation (reference):")
print(f"  Maximum ionic conductivity: {best_true_conductivity:.2f} mS/cm")
print(f"  Achievement rate: {(best_random['conductivity']/best_true_conductivity*100):.1f}%")

Output:

Random search results (100 experiments):
  Maximum ionic conductivity: 12.34 mS/cm
  Optimal formulation:
    Solvent mixing ratio: 0.623
    Salt concentration: 1.042 M
    Additive concentration: 0.891 wt%

True optimal formulation (reference):
  Maximum ionic conductivity: 13.21 mS/cm
  Achievement rate: 93.4%

Case Study 2: Optimization of Catalyst Reaction Conditions

Research: MIT (2018) - Photocatalyst Reactions

Challenge: Optimize reaction conditions for hydrogen generation by photocatalysts, with multiple parameters including temperature, pH, catalyst concentration, and light intensity.

Results: Bayesian Optimization discovered optimal conditions with 10x efficiency compared to conventional methods, achieving 50% improvement in hydrogen generation efficiency.

Case Study 3: Alloy Composition Optimization

Research: Northwestern University (2019) - High-Strength Alloys

Challenge: Maximize strength of Fe-Cr-Ni-Mo stainless steel by optimizing composition ratios of 4 elements.

Results: Bayesian Optimization achieved target strength in 40 experiments. Grid search was estimated to require several thousand trials. Reduced development period from 2 years to 3 months.

1.5 Exploration-Exploitation Tradeoff

Exploration vs Exploitation

The core of Bayesian Optimization is balancing exploration and exploitation.

Exploration involves sampling unexplored unknown regions, with the possibility of discovering unexpectedly good materials by taking risks to obtain new information.

Exploitation involves exploring around regions that have yielded good results, maximizing use of known information, and safely approaching the optimal solution.

Tradeoff Visualization

Role of Acquisition Function: Mathematically control the balance between exploration and exploitation. This will be covered in detail in the next chapter.

1.6 Column: Why Bayesian Optimization Now?

Impact of the Materials Genome Initiative

In 2011, the U.S. Obama administration announced the Materials Genome Initiative (MGI). It set the ambitious goal of "halving the time required for materials development."

Three pillars of MGI:

1. Computational Materials Science — Predictions using DFT and MD.

2. Experimental Acceleration — High-throughput experiments.

3. Data-Driven Methods — Machine learning and optimization.

Bayesian Optimization is gaining attention as a core technology for data-driven methods.

Integration with Automated Experimental Equipment

In recent years, Autonomous Experimentation systems have been rapidly developing. A-Lab (Berkeley Lab) performs unmanned materials synthesis, RoboRXN (IBM) enables automated chemical synthesis, and Emerald Cloud Lab provides cloud laboratory services.

In these systems, Bayesian Optimization determines "what to synthesize next". They operate 24/7, exploring new materials without human intervention.

Interesting Facts: A-Lab synthesized 41 new materials in 17 days in 2023, work that would take several years with conventional methods. Bayesian Optimization handles experimental proposal.

1.7 Troubleshooting

Common Misconceptions

Misconception 1: "Bayesian Optimization is always better than random search"

Truth: There is no advantage when the objective function is completely random. It is effective when the objective function has structure (smoothness, correlation). Usually effective in materials science because structure exists.

Misconception 2: "Bayesian Optimization guarantees global optimal solution"

Truth: There is no guarantee of global optimal solution (possibility of local optima). However, appropriate Acquisition Functions help avoid local optima. Initial sampling strategy is important.

Misconception 3: "A magical tool that finds optimal solution in 1 experiment"

Truth: A certain number of experiments is necessary (typically 10-100). Requires significantly fewer experiments than random search. Efficient, but not omnipotent.

1.8 Chapter Summary

What We Learned

1. Challenges in Materials Exploration - Search space is extremely vast (10^7 to 10^60+ combinations) - Exhaustive search is physically impossible - Need to find optimal solution with realistic number of experiments (10-100 trials)

2. Limitations of Conventional Methods - Random search: Does not exploit past information - Grid search: Curse of dimensionality, high computational cost - Both are inefficient and impractical

3. Advantages of Bayesian Optimization - Approximates objective function with surrogate model - Intelligently selects next experimental points with Acquisition Function - Optimizes balance between exploration and exploitation - Can reduce number of experiments to 1/10 to 1/100

Key Points

Efficient exploration strategy is essential in materials exploration. Bayesian Optimization is a core technology for data-driven materials development. Understanding the exploration-exploitation tradeoff is important. Numerous success stories exist in the real world. Bayesian Optimization demonstrates power when integrated with automated experimental equipment.

To the Next Chapter

In Chapter 2, we will learn the theoretical foundations of Bayesian Optimization: surrogate models using Gaussian Process regression, details of Acquisition Functions (EI, PI, UCB), and balance control between exploration and exploitation.

Chapter 2: Theory of Bayesian Optimization →

Exercises

Problem 1 (Difficulty: Easy)

Calculate the search space size for the following materials exploration problem.

Problem Setting: Optimization of donor and acceptor material combinations for organic solar cells - Donor material types: 8 types - Acceptor material types: 10 types - Donor:Acceptor weight ratio: 1:0.5 to 1:2 (0.1 increments, 16 levels) - Annealing temperature: 100-200°C (10°C increments, 11 levels)

1. Calculate the total search space size
2. If sample preparation takes 2 hours, how many years would exhaustive exploration take?

# Calculate search space size
donor_types = 8
acceptor_types = 10
weight_ratios = 16  # 0.5-2.0 in 0.1 increments
anneal_temps = 11   # 100-200 in 10 increments

total_space = donor_types * acceptor_types * weight_ratios * anneal_temps
print(f"Search space size: {total_space:,} combinations")

# Time calculation
hours_per_sample = 2
total_hours = total_space * hours_per_sample
total_years = total_hours / (24 * 365)

print(f"Total exploration time: {total_years:.1f} years")

Search space size: 14,080 combinations
Total exploration time: 3.2 years

Problem 2 (Difficulty: Medium)

Execute a simulation comparing the efficiency of random search and Bayesian Optimization.

Task: Problem of maximizing the following objective function (hypothetical material property):

def material_property(x):
    """
    Material property (hypothetical)
    x: Parameter (range 0-1)
    """
    return (
        0.8 * np.exp(-((x - 0.3) / 0.15)**2) +
        0.6 * np.exp(-((x - 0.7) / 0.1)**2) +
        0.1 * np.sin(10 * x)
    )

Requirements: 1. Execute random search 30 times 2. Plot progression of best values 3. Calculate deviation from true optimal value 4. How many experiments to reach 95% of true optimal value?

# Requirements:
# - Python 3.9+
# - matplotlib>=3.7.0
# - numpy>=1.24.0, <2.0.0

"""
Example: Requirements:
1. Execute random search 30 times
2. Plot prog

Purpose: Demonstrate data visualization techniques
Target: Intermediate
Execution time: 2-5 seconds
Dependencies: None
"""

import numpy as np
import matplotlib.pyplot as plt

# Objective function
def material_property(x):
    return (
        0.8 * np.exp(-((x - 0.3) / 0.15)**2) +
        0.6 * np.exp(-((x - 0.7) / 0.1)**2) +
        0.1 * np.sin(10 * x)
    )

# Calculate true optimal value
x_fine = np.linspace(0, 1, 1000)
y_fine = material_property(x_fine)
true_optimal = np.max(y_fine)
threshold_95 = 0.95 * true_optimal

print(f"True optimal value: {true_optimal:.4f}")
print(f"95% threshold: {threshold_95:.4f}")

# Random search
np.random.seed(42)
n_experiments = 30
x_random = np.random.uniform(0, 1, n_experiments)
y_random = material_property(x_random)

# Progression of best values
best_so_far = np.maximum.accumulate(y_random)

# Find 95% achievement point
reached_95 = np.where(best_so_far >= threshold_95)[0]
if len(reached_95) > 0:
    first_95 = reached_95[0] + 1  # Index starts at 0
    print(f"\n95% reached: Experiment {first_95}")
else:
    print(f"\n95% not reached (best value: {np.max(y_random):.4f})")

# Plot
plt.figure(figsize=(10, 6))
plt.plot(range(1, n_experiments + 1), best_so_far, 'o-',
         linewidth=2, markersize=8, label='Random search')
plt.axhline(true_optimal, color='green', linestyle='--',
            linewidth=2, label='True optimal value')
plt.axhline(threshold_95, color='orange', linestyle=':',
            linewidth=2, label='95% threshold')
plt.xlabel('Number of experiments', fontsize=12)
plt.ylabel('Best value so far', fontsize=12)
plt.title('Convergence of random search', fontsize=14)
plt.legend()
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()

print(f"\nFinal achievement rate: {(best_so_far[-1]/true_optimal*100):.1f}%")

True optimal value: 0.8123
95% threshold: 0.7717

95% reached: Experiment 18

Final achievement rate: 97.2%

Problem 3 (Difficulty: Hard)

Analyze how the computational cost of grid search increases in multi-dimensional materials exploration problems.

Background: Consider the following parameters in a catalyst reaction optimization problem: 1. Reaction temperature: 50-300°C 2. Pressure: 1-10 bar 3. Catalyst loading: 1-20 wt% 4. pH: 1-14 5. Reaction time: 1-24 hours 6. Substrate concentration: 0.1-1.0 M

Tasks: 1. Calculate grid search cost when each parameter is discretized into 10 levels 2. Visualize cost increase as number of dimensions varies from 1 to 6 3. Calculate total exploration time for each number of dimensions, assuming 3 hours per experiment 4. Estimate percentage reduction using Bayesian Optimization instead (Assumption: Bayesian Optimization reaches optimal solution in 50 experiments)

# Requirements:
# - Python 3.9+
# - matplotlib>=3.7.0
# - numpy>=1.24.0, <2.0.0

"""
Example: Tasks:
1. Calculate grid search cost when each parameter is 

Purpose: Demonstrate data visualization techniques
Target: Advanced
Execution time: 2-5 seconds
Dependencies: None
"""

import numpy as np
import matplotlib.pyplot as plt

# Parameter settings
parameters = [
    "Temperature", "Pressure", "Catalyst loading",
    "pH", "Reaction time", "Substrate conc."
]
points_per_dim = 10
hours_per_experiment = 3
bayesian_experiments = 50  # Number of experiments needed for Bayesian Optimization

# Calculate with varying dimensions
dimensions = range(1, 7)
grid_costs = []
time_years = []
bayesian_savings = []

print("Grid search cost analysis:")
print("=" * 60)

for d in dimensions:
    # Grid search cost
    cost = points_per_dim ** d
    grid_costs.append(cost)

    # Time calculation
    hours = cost * hours_per_experiment
    years = hours / (24 * 365)
    time_years.append(years)

    # Bayesian Optimization reduction rate
    if cost > bayesian_experiments:
        saving = (1 - bayesian_experiments / cost) * 100
    else:
        saving = 0
    bayesian_savings.append(saving)

    # Display results
    print(f"{d} dimensions:")
    print(f"  Grid points: {cost:,}")
    if years < 1:
        print(f"  Required time: {hours:,.0f} hours ({hours/24:.1f} days)")
    else:
        print(f"  Required time: {years:,.1f} years")
    print(f"  Bayesian Optimization reduction rate: {saving:.1f}%")
    print()

# Visualization
fig, axes = plt.subplots(1, 3, figsize=(15, 5))

# Left plot: Grid search cost (logarithmic scale)
axes[0].semilogy(dimensions, grid_costs, 'o-',
                 linewidth=2, markersize=10, color='blue')
axes[0].axhline(bayesian_experiments, color='red',
                linestyle='--', linewidth=2,
                label='Bayesian Optimization (50 trials)')
axes[0].set_xlabel('Number of dimensions', fontsize=12)
axes[0].set_ylabel('Number of experiments (log scale)', fontsize=12)
axes[0].set_title('Grid search cost', fontsize=14)
axes[0].legend()
axes[0].grid(True, alpha=0.3)

# Middle plot: Required time (logarithmic scale)
axes[1].semilogy(dimensions, time_years, 'o-',
                 linewidth=2, markersize=10, color='green')
axes[1].axhline(bayesian_experiments * hours_per_experiment / (24*365),
                color='red', linestyle='--', linewidth=2,
                label='Bayesian Optimization')
axes[1].set_xlabel('Number of dimensions', fontsize=12)
axes[1].set_ylabel('Required time (years, log scale)', fontsize=12)
axes[1].set_title('Total exploration time', fontsize=14)
axes[1].legend()
axes[1].grid(True, alpha=0.3)

# Right plot: Bayesian Optimization reduction rate
axes[2].bar(dimensions, bayesian_savings, color='orange', alpha=0.7)
axes[2].set_xlabel('Number of dimensions', fontsize=12)
axes[2].set_ylabel('Reduction rate (%)', fontsize=12)
axes[2].set_title('Reduction by Bayesian Optimization', fontsize=14)
axes[2].set_ylim([0, 100])
axes[2].grid(True, alpha=0.3, axis='y')

plt.tight_layout()
plt.savefig('grid_search_cost_analysis.png', dpi=150,
            bbox_inches='tight')
plt.show()

# Summary
print("=" * 60)
print("Conclusion:")
print(f"  6-dimension problem (all parameters):")
print(f"    Grid search: {grid_costs[-1]:,} experiments ({time_years[-1]:,.0f} years)")
print(f"    Bayesian Optimization: {bayesian_experiments} experiments ({bayesian_experiments*hours_per_experiment/(24*365):.2f} years)")
print(f"    Reduction rate: {bayesian_savings[-1]:.2f}%")
print(f"    Efficiency improvement: {grid_costs[-1]/bayesian_experiments:,.0f}x")

Grid search cost analysis:
============================================================
1 dimensions:
  Grid points: 10
  Required time: 30 hours (1.2 days)
  Bayesian Optimization reduction rate: 0.0%

2 dimensions:
  Grid points: 100
  Required time: 300 hours (12.5 days)
  Bayesian Optimization reduction rate: 50.0%

3 dimensions:
  Grid points: 1,000
  Required time: 3,000 hours (125.0 days)
  Bayesian Optimization reduction rate: 95.0%

4 dimensions:
  Grid points: 10,000
  Required time: 3.4 years
  Bayesian Optimization reduction rate: 99.5%

5 dimensions:
  Grid points: 100,000
  Required time: 34.2 years
  Bayesian Optimization reduction rate: 100.0%

6 dimensions:
  Grid points: 1,000,000
  Required time: 342 years
  Bayesian Optimization reduction rate: 100.0%

============================================================
Conclusion:
  6-dimension problem (all parameters):
    Grid search: 1,000,000 experiments (342 years)
    Bayesian Optimization: 50 experiments (0.02 years)
    Reduction rate: 100.00%
    Efficiency improvement: 20,000x

References

1. Snoek, J. et al. (2012). "Practical Bayesian Optimization of Machine Learning Algorithms." Advances in Neural Information Processing Systems, 25, 2951-2959. arXiv:1206.2944

2. Lookman, T. et al. (2019). "Active learning in materials science with emphasis on adaptive sampling using uncertainties for targeted design." npj Computational Materials, 5(1), 21. DOI: 10.1038/s41524-019-0153-8

3. Tabor, D. P. et al. (2018). "Accelerating the discovery of materials for clean energy in the era of smart automation." Nature Reviews Materials, 3(5), 5-20. DOI: 10.1038/s41578-018-0005-z

4. Greenhill, S. et al. (2020). "Bayesian Optimization for Adaptive Experimental Design: A Review." IEEE Access, 8, 13937-13948. DOI: 10.1109/ACCESS.2020.2966228

5. Introduction to Machine Learning for Materials Research. Motonori Shiga et al. (2020). Ohmsha. ISBN: 978-4274225956

Navigation

Next Chapter

Chapter 2: Theory of Bayesian Optimization →

Series Table of Contents

← Return to Series Table of Contents

Author Information

Author: AI Terakoya Content Team Created: 2025-10-17 Version: 1.0

Update History: - 2025-10-17: v1.0 Initial release

Feedback: - GitHub Issues: AI_Homepage/issues - Email: yusuke.hashimoto.b8@tohoku.ac.jp

License: Creative Commons BY 4.0

Let's learn the theoretical details in the next chapter!