Introduction to Process Informatics Series v1.0
The future of chemical process optimization opened by data - A complete guide from history to practice and careers
Series Overview
This series is educational content structured in four chapters that can be learned step by step, from those studying Process Informatics (PI) for the first time to those who want to acquire practical skills.
Features:
- ✅ Independence of each chapter: Each chapter can be read as a standalone article
- ✅ Systematic structure: Comprehensive content that can be learned step by step across all four chapters
- ✅ Practice-oriented: 35 executable code examples and 5 detailed case studies
- ✅ Career support: Provides concrete career paths and learning roadmaps
Total learning time: 90-120 minutes (including code execution and exercises)
How to Proceed with Learning
Recommended Learning Order
For beginners (completely new):
- Chapter 1 → Chapter 2 → Chapter 3 (some parts can be skipped) → Chapter 4
- Time required: 70-90 minutes
For those with Python experience (with basic knowledge):
- Chapter 2 → Chapter 3 → Chapter 4
- Time required: 60-80 minutes
For strengthening practical skills (already familiar with PI concepts):
- Chapter 3 (intensive study) → Chapter 4
- Time required: 50-65 minutes
Details of Each Chapter
Chapter 1: Why Process Informatics
Difficulty: Introductory
Reading time: 15-20 minutes
Learning Content
-
The history of chemical process development
- From ancient distillation techniques to modern process control
- Evolution of development methods: trial and error → rules of thumb → theory-driven → data-driven -
Limitations of conventional methods
- Time: 1-3 years for scale-up
- Cost: tens of billions of yen for plant construction
- Batch-to-batch variation: quality inconsistency problems -
Detailed case study of chemical plant optimization
- Case of improving yield from 70% → 85%
- 30% reduction in energy consumption
- Development period can be shortened to 1/3 using PI -
Comparison diagram (Traditional vs PI)
- Mermaid diagram: visualization of the workflow
- Timing comparison: 6 months/condition vs 1 week/condition -
Column: "A Day in the Life"
- Process engineer in 1990: 1 experiment/week, manual data analysis
- Process engineer in 2025: 50 experiments/week (automated), AI-driven optimization proposals -
The three convergence factors of "Why Now?"
- Sensor technology: IoT, real-time monitoring
- Data infrastructure: cloud, big data processing
- Social urgency: carbon neutrality, quality assurance, DX
Learning Objectives
- ✅ Explain the historical evolution of chemical process development
- ✅ Cite three limitations of conventional methods with concrete examples
- ✅ Understand the social and technical background driving the demand for PI
Chapter 2: PI Fundamentals - Concepts, Methods, and Ecosystem
Difficulty: Introductory to Intermediate
Reading time: 20-25 minutes
Learning Content
-
Definition of PI and related fields
- The etymology and history of Process Informatics
- Relationship with Industry 4.0 and smart factories
- Differences from Quality Engineering (QE) and Design of Experiments (DoE) -
Glossary of 20 PI terms
- 3 categories: basic terms, method terms, application terms
- Each term: Japanese, English, and a 1-2 sentence explanation -
Types of key process data
- Process parameters: temperature, pressure, flow rate, residence time
- Product properties: yield, selectivity, purity, quality indicators
- Operation data: energy consumption, equipment status -
PI ecosystem diagram
- Mermaid diagram: sensors → data collection → ML → optimization → process control
- Visualization of the feedback loop -
5-step workflow (detailed version)
- Step 0: Problem formulation (improve yield? reduce cost?)
- Step 1: Data collection (time-series data, experimental data)
- Step 2: Model building (regression, classification, time-series prediction)
- Step 3: Optimization (Bayesian optimization, multi-objective optimization)
- Step 4: Implementation and validation (pilot scale, actual plant)
- Each step: sub-steps, common pitfalls, time estimates -
Deep dive into process descriptors
- Physicochemical parameters: concentration, temperature, pressure, pH
- Equipment characteristics: reactor size, stirring speed, residence time
- Operating conditions: feed rate, heating rate, cooling rate
Learning Objectives
- ✅ Explain the definition of PI and its differences from other fields (QE, DoE, etc.)
- ✅ Understand the main data types of chemical processes
- ✅ Describe the 5 steps of the PI workflow down to each sub-step
- ✅ Explain the types of process descriptors with examples
- ✅ Use 20 PI technical terms appropriately
Chapter 3: Experiencing PI with Python - Process Optimization in Practice
Difficulty: Intermediate
Reading time: 30-40 minutes
Code examples: 35 (all executable)
Learning Content
-
Environment setup (three options)
- Option 1: Anaconda (recommended for beginners, includes GUI)
- Option 2: venv (Python standard, lightweight)
- Option 3: Google Colab (no installation required, cloud-based) -
Six machine learning models (complete implementation)
- Example 1: Linear Regression (yield prediction, R²=0.75)
- Example 2: Random Forest (yield and selectivity prediction, R²=0.88)
- Example 3: LightGBM (gradient boosting, R²=0.91)
- Example 4: SVR (nonlinear process optimization, R²=0.86)
- Example 5: Time Series Analysis (ARIMA, Prophet)
- Example 6: Bayesian Optimization (reaction condition optimization) -
Model performance comparison
- Comparison table: MAE, R², training time, interpretability
- Visualization: bar charts for each metric
- Model selection flowchart (Mermaid diagram) -
Process optimization methods
- Grid Search: exhaustive search (temperature × pressure × concentration)
- Bayesian Optimization: efficient search (optimal conditions in 10-20 experiments)
- Multi-objective Optimization: trade-off between yield and cost -
Feature engineering
- Interaction terms of process parameters
- Time-series features (moving averages, lag variables)
- Derived variables for quality indicators -
Troubleshooting guide
- 7 common errors and their solutions (in table form)
- 5-step debugging checklist
- Performance improvement strategies -
Project challenge
- Goal: Yield optimization of a chemical reactor (yield > 80%)
- 6-step guide:- Process data collection
- Data preprocessing (outlier removal)
- Feature engineering
- Model training (Random Forest recommended)
- Condition search using Bayesian optimization
- Results visualization (response surface plot, Pareto front)
Learning Objectives
- ✅ Set up a Python environment using one of three methods
- ✅ Implement 6 types of machine learning models and compare their performance
- ✅ Execute Bayesian optimization
- ✅ Perform multi-objective optimization (yield vs cost)
- ✅ Troubleshoot common errors on your own
- ✅ Complete a chemical reactor optimization project
Chapter 4: Real-World Applications of PI - Success Stories and Future Outlook
Difficulty: Intermediate to Advanced
Reading time: 20-25 minutes
Learning Content
- Five detailed case studies
Case Study 1: Catalytic process optimization (yield improvement)
- Technology: Bayesian Optimization, Random Forest
- Results: yield 70% → 85% (+15 pt), development period 6 months → 2 months
- Impact: annual revenue increase of 2 billion yen
- Company: Chemical Manufacturer A
Case Study 2: Polymerization reaction control (molecular weight distribution)
- Technology: Time Series Analysis, combined PID control and ML
- Results: 50% reduction in standard deviation of molecular weight distribution, defect rate 5% → 1%
- Impact: annual disposal cost reduction of 500 million yen
- Company: Polymer Manufacturer B
Case Study 3: Distillation column optimization (energy reduction)
- Technology: Multi-objective Optimization, Soft Sensor
- Results: 30% reduction in energy consumption, maintaining 99.5% purity
- Impact: reduced CO2 emissions, annual utility cost reduction of 300 million yen
- Company: Petrochemical Manufacturer C
Case Study 4: Pharmaceutical batch process (quality consistency)
- Technology: Statistical Process Control (SPC), DoE + ML
- Results: 70% reduction in batch-to-batch variation, 100% regulatory compliance achieved
- Impact: passed FDA inspection on the first attempt, 3-month reduction in time to market
- Company: Pharmaceutical Manufacturer D
Case Study 5: Bioprocess optimization (fermentation)
- Technology: Online Learning, metabolic model + ML
- Results: +40% cell concentration, +50% productivity
- Impact: 30% reduction in biofuel cost, contribution to carbon neutrality
- Company: Biotechnology Company E
- Future trends (three major trends)
Trend 1: Digital Twin
- Example: real-time process simulation
- Prediction: 80% of major chemical companies will adopt it by 2030
- Initial investment: 500 million yen, ROI: recovered in 1-2 years
Trend 2: Autonomous Control
- Example: 24/7/365 optimization by AI
- Effect: 80% reduction in human operator intervention
- Prediction: 20% improvement in operational efficiency by 2030
Trend 3: Sustainability DX
- LCA integration: carbon footprint optimization
- Example: green chemistry (30% solvent reduction)
- Example: byproduct reuse (zero emissions)
- Career paths (three main routes)
Path 1: Academia (Researcher)
- Route: Bachelor's → Master's → PhD (3-5 years) → Postdoc (2-3 years) → Associate Professor
- Salary: annual income 5-12 million yen (Japan)
- Skills: Python, ML, chemical engineering, paper writing
- Examples: University of Tokyo, Kyoto University, MIT
Path 2: Industry R&D
- Positions: process engineer, data scientist
- Salary: annual income 7-15 million yen (Japan)
- Companies: Mitsubishi Chemical, Asahi Kasei, Sumitomo Chemical, BASF
- Skills: Python, ML, process control, project management
Path 3: Startup / DX Consulting
- Example: process DX consulting firm
- Salary: annual income 6-12 million yen + performance-based compensation
- Risk/return: high risk, high impact
- Required skills: technology + business + consulting
-
Skill development timeline
- 3-month plan: fundamentals → practice → portfolio
- 1-year plan: advanced ML → projects → conference presentations
- 3-year plan: expert → paper publication → leadership -
Collection of learning resources
- Online courses: Coursera, edX, Udemy (specific course names)
- Books: "Process Systems Engineering" by Seborg, etc.
- Communities: The Society of Chemical Engineers, Japan (SCEJ), AIChE
- Conferences: PSE, ESCAPE, SCEJ Annual Meeting
Learning Objectives
- ✅ Explain five real-world PI success stories with technical details
- ✅ List three future PI trends and evaluate their impact on industry
- ✅ Explain three types of career paths in the PI field and understand the required skills
- ✅ Plan a concrete learning timeline (3 months / 1 year / 3 years)
- ✅ Select appropriate learning resources as the next step
Overall Learning Outcomes
Upon completing this series, you will acquire the following skills and knowledge:
Understanding
- ✅ Explain the historical background and necessity of PI
- ✅ Understand the basic concepts, terminology, and methods of PI
- ✅ Grasp the types and handling of process data
- ✅ Describe five or more real-world success stories in detail
Practical Skills (Doing)
- ✅ Set up a Python environment and install the necessary libraries
- ✅ Implement 6 types of machine learning models and compare their performance
- ✅ Optimize process conditions with Bayesian optimization
- ✅ Perform multi-objective optimization (yield vs cost)
- ✅ Analyze and predict time-series data
- ✅ Debug errors on your own
Application Ability (Applying)
- ✅ Design a new chemical process optimization project
- ✅ Evaluate industrial adoption cases and apply them to your own research
- ✅ Concretely plan your future career path
- ✅ Formulate a continuous learning strategy
Recommended Learning Patterns
Pattern 1: Complete Mastery (for beginners)
Target: Those learning PI for the first time, those who want to understand it systematically
Duration: 2-3 weeks
How to proceed:
Week 1:
- Day 1-2: Chapter 1 (History and background)
- Day 3-4: Chapter 2 (Fundamentals)
- Day 5-7: Chapter 2 exercises, terminology review
Week 2:
- Day 1-3: Chapter 3 (Python environment setup)
- Day 4-5: Chapter 3 (Implement models 1-3)
- Day 6-7: Chapter 3 (Implement models 4-6)
Week 3:
- Day 1-2: Chapter 3 (Project challenge)
- Day 3-4: Chapter 4 (Case studies)
- Day 5-7: Chapter 4 (Create career plan)
Deliverables:
- Chemical reactor optimization project (yield > 80%)
- Personal career roadmap (3 months / 1 year / 3 years)
Pattern 2: Fast Track (for those with Python experience)
Target: Those with basic knowledge of Python and machine learning
Duration: 1 week
How to proceed:
Day 1: Chapter 2 (focusing on PI-specific concepts)
Day 2-3: Chapter 3 (implement all code)
Day 4: Chapter 3 (Project challenge)
Day 5-6: Chapter 4 (Case studies and careers)
Day 7: Review and plan next steps
Deliverables:
- Performance comparison report of 6 models
- Project portfolio (public release on GitHub recommended)
Pattern 3: Pinpoint Learning (focused on specific topics)
Target: Those who want to strengthen a specific skill or knowledge
Duration: Flexible
Selection examples:
- Want to learn process data analysis → Chapter 2 (Section 2.3-2.4) + Chapter 3 (Example 5)
- Want to master Bayesian optimization → Chapter 3 (Section 3.4)
- Want to design a career → Chapter 4 (Section 4.3-4.4)
- Want to know the latest trends → Chapter 4 (Section 4.2)
FAQ (Frequently Asked Questions)
Q1: Can programming beginners understand it?
A: Chapters 1 and 2 are theory-centered, so no programming experience is required. Chapter 3 assumes you understand basic Python syntax (variables, functions, lists), but the code examples are thoroughly commented, so even beginners can learn step by step. If you are unsure, we recommend learning the basics with the Python Introductory Tutorial before Chapter 3.
Q2: Which chapter should I start with?
A: For first-timers, we strongly recommend reading from Chapter 1 in order. Although each chapter is independent, the concepts are designed to build on one another. If you have Python experience and limited time, you can also start from Chapter 2.
Q3: Do I need to actually run the code?
A: To maximize the learning effect of Chapter 3, we strongly recommend actually running the code. Understanding differs greatly between just reading and actually executing. If setting up the environment is difficult, start with Google Colab (free, no installation required).
Q4: How long does it take to master?
A: It depends on your study time and goals:
- Conceptual understanding only: 1-2 days (Chapters 1 and 2)
- Basic implementation skills: 1-2 weeks (Chapters 1-3)
- Practical project execution ability: 2-4 weeks (all four chapters + project challenge)
- Professional-level skills: 3-6 months (series completion + additional projects)
Q5: Can I become a PI expert with this series alone?
A: This series targets "introductory to intermediate" levels. To reach an expert level:
1. Solidify the fundamentals with this series (2-4 weeks)
2. Learn advanced content from the learning resources in Chapter 4 (3-6 months)
3. Carry out your own projects (6-12 months)
4. Present at conferences and write papers (1-2 years)
A total of 2-3 years of continuous study and practice is required.
Q6: Can it be applied in languages other than Python (MATLAB, R, etc.)?
A: Since the principles and methods are language-independent, they are theoretically applicable. However:
- Python is mainstream in the PI field (libraries: scikit-learn, pandas, scipy)
- MATLAB has a track record in process control (Simulink)
- Learning resources are also Python-centered
Recommendation: We recommend becoming proficient in Python.
Q7: Are the exercises in each chapter mandatory?
A: They are not mandatory, but we strongly recommend them to confirm your understanding. The exercises let you:
- Review the key points of the chapter
- Cultivate practical application ability
- Notice misunderstandings or gaps in understanding
If you are short on time, please at least solve the "easy" problems in each chapter.
Q8: Is it okay if I don't have a chemical engineering background?
A: Basically, it is desirable to have basic knowledge of chemical engineering:
- Reaction kinetics, mass balance, energy balance
- Basics of process control
However, since this series explains the necessary concepts, it is designed so that even beginners can understand it. If you want to learn the details of chemical engineering, please read the reference textbooks alongside it.
Q9: Is there a community where I can ask questions or discuss?
A: You can ask questions and discuss in the following communities:
- Japan: The Society of Chemical Engineers, Japan (SCEJ), Japan Process Engineering Association
- International: AIChE (American Institute of Chemical Engineers)
- Online:
- Stack Overflow (process-optimization, chemical-engineering tags)
- LinkedIn Process Engineering Groups. This theme is broad,
Next Steps
Recommended Actions After Completing the Series
Immediate (within 1-2 weeks):
1. ✅ Create a portfolio on GitHub/GitLab
2. ✅ Publish the results of the project challenge with a README
3. ✅ Add the "Process Informatics" skill to your LinkedIn profile
Short-term (1-3 months):
1. ✅ Choose one of the learning resources from Chapter 4 and dive deep
2. ✅ Participate in a Kaggle chemical process competition
3. ✅ Attend study meetings of the SCEJ / AIChE
4. ✅ Carry out your own small-scale project (e.g., optimization of a specific process)
Medium-term (3-6 months):
1. ✅ Carefully read 10 papers (Computers & Chemical Engineering, Industrial & Engineering Chemistry Research)
2. ✅ Contribute to open-source projects (scipy, pandas, etc.)
3. ✅ Present at a domestic conference (poster or oral)
4. ✅ Participate in an internship or collaborative research
Long-term (1 year or more):
1. ✅ Present at international conferences (PSE, ESCAPE)
2. ✅ Submit a peer-reviewed paper
3. ✅ Get a job related to PI (academia or industry)
4. ✅ Train the next generation of PI engineers
Feedback and Support
About This Series
This series was created under Dr. Yusuke Hashimoto of Tohoku University as part of the MI Knowledge Hub project.
Creation date: October 16, 2025
Version: 1.0
We Welcome Your Feedback
To improve this series, we welcome your feedback:
- Typos, omissions, and technical errors: Report via an Issue in the GitHub repository
- Improvement suggestions: New topics, code examples you would like added, etc.
- Questions: Parts that were difficult to understand, sections where you want additional explanation
- Success stories: Projects using what you learned from this series
Contact: yusuke.hashimoto.b8@tohoku.ac.jp
License and Terms of Use
This series is published under the CC BY 4.0 (Creative Commons Attribution 4.0 International) license.
What you can do:
- ✅ Freely view and download
- ✅ Use for educational purposes (classes, study meetings, etc.)
- ✅ Modify and create derivative works (translation, summary, etc.)
Conditions:
- 📌 Author credit is required
- 📌 If modified, state that clearly
- 📌 For commercial use, please contact us in advance
Details: Full text of the CC BY 4.0 license
Now, Let's Get Started!
Are you ready? Start with Chapter 1 and begin your journey into the world of PI!
Chapter 1: Why Process Informatics →
Update History
- 2025-10-16: v1.0 first edition published
Your PI learning journey starts here!