Capstone Mini-Project: Build a Property-Prediction Model on Your Own Data

This page is not a tutorial you follow step by step. It is an assignment template for choosing your own data, designing your own approach, and finishing the loop yourself. The Quickstart (predicting band gap from composition) is your reference implementation. Work through the Quickstart once, then run the same loop on your own data. Each section provides a checklist, milestones, a self-assessment rubric, and a report template.

🧭 Where this assignment fits

This is a capstone for those who have finished Stages 1–2 of the roadmap (the big picture and core techniques). If you have gone as far as Stages 3–4, you can attempt a more advanced design. You may reuse the reference implementation's code directly; the point is to take the step from "reading executed code" to "running the loop yourself."

Reference implementation (Quickstart) → 🗺️ Back to the roadmap

🎯 Goal

In this assignment, you will complete the following on your own.

On 30 or more samples of materials data, run the full sequence: featurization to a baseline to cross-validation to interpretation to a limits discussion, all the way through.
Write up the process and results in a short report of about two to three A4 pages (a template is provided near the bottom of this page).
The rough time estimate is four to eight hours. If gathering data takes a while, starting from public data or extending the reference implementation shortens it.

The goal is not to hit perfect accuracy. The attitude this project aims to build is to evaluate honestly, interpret the results in your own words, and state the limits plainly.

🗂️ Choosing your data

Pick one of the three entry points below. You can start even without data on hand. Whichever you choose, always record the source and the date you obtained it.

A. Your own research or work data

The most rewarding entry point. Reframe the measurements or synthesis conditions you handle every day as a prediction task.

Decide the target variable (the property to predict) and the inputs (composition, conditions, and so on) first.
Even for data you cannot publish, the report only needs the count and the range.

B. Use public data

The standard route when you have no data of your own. Pull a property that interests you from one of the following.

Materials Project (DFT-computed properties)
MatBench (a suite of regression / classification benchmarks)
OQMD (formation energies and more)
NOMAD (diverse computed and experimental data)

C. Extend the reference data

For the fastest start. Add your own compounds to the 30 in the Quickstart.

Add 10–20 compounds from the literature to reach 40–50 entries.
Append the measured, tabulated values for any new elements to the element-property table.

✅ Required-criteria checklist

Before you submit your report (submitting to yourself is fine), satisfy all seven items below. Your checkbox state is saved in this browser and is not shared with anyone.

Status 0 / 7 criteria met

At least 30 entries, with the source stated Record the count, the scope of materials, the source (a reference or database name), and the date obtained. Do not cherry-pick values to make the data look tidy.
Composition- or structure-based featurization, with your reasons Note which descriptors you used and why you expected them to relate to the target. The reference implementation's six descriptors (electronegativity difference and so on) make a fine starting point.
A baseline (mean prediction or dummy regression) Build a "learns nothing" reference with scikit-learn's DummyRegressor or similar, and confirm the model truly beats it.
Cross-validation suited to the data size, with your rationale Explain your choice against the count: Leave-One-Out for a few dozen entries, 5-fold once you have a hundred or more.
MAE and R² reported Show the mean absolute error (MAE) and coefficient of determination (R²) from cross-validation, alongside the baseline's values.
Interpretation via feature importance or error analysis Examine which features mattered and which samples were badly missed, and give them meaning in the language of materials science.
A discussion of limits and the next move State plainly where the limits lie — in data size, features, or method — and what you would try next.

📅 Milestones

Splitting the work into four stages makes the whole easier to see. The times are estimates.

What each stage involves and how you know it is done
Stage	Main work	Done when
1. Data prep	Select, gather, and clean the data. Fix the target and inputs, and record the source.	You have a table of 30+ entries, can explain each row's source, and have checked for missing values and duplicates.
2. Model building	Implement featurization, a baseline, and a regression model (start with random forest or similar).	You can build the feature matrix and train both the baseline and the main model.
3. Evaluation	Compute MAE and R² by cross-validation. Inspect predictions one by one and extract feature importance.	You can compare against the baseline and name specific hits and misses.
4. Report	Following the template below, write up background through limits in two to three pages.	All six template items are filled, and a reader could follow how to reproduce your work.

📊 Self-assessment rubric

Before submitting, review your work along five dimensions. Anything that lands in "Needs work" is where you can grow next.

A self-assessment table of dimensions by three levels
Dimension	Advanced	Standard	Needs work
Data quality	Source, date obtained, and known biases are all stated, with preprocessing decisions recorded.	Count and source are stated, and basic preprocessing is done.	The source is vague, or values are cherry-picked.
Method choice	The choices of features, model, and CV share a consistent, data-driven rationale.	Reasonable features and a model are chosen, and CV is set up.	No reasons are given, and defaults are used as-is.
Soundness of evaluation	Compared against a baseline, with a check that there is no data leakage.	MAE and R² are reported via cross-validation.	Evaluated on the training data, or metrics are missing.
Depth of interpretation	Importance and errors are tied back to known physics.	The effective features and the missed samples are pointed out.	Numbers are listed without any meaning attached.
Clarity of reporting	A reader can follow how to reproduce it, and limits and next moves are stated plainly.	The template items are filled and the narrative can be followed.	Items are missing, and the conclusion or limits cannot be read off.

📝 Report template

Use the six items below directly as headings. A few lines per item is enough. Filling them in reveals the gaps in your own understanding.

# Property-Prediction Mini-Project Report

## 1. Background and objective
- The property to predict, and why it matters
- The question you want to test (e.g., how far composition alone can go)

## 2. Data
- Source (reference or database name, date obtained)
- Count, scope of materials, preprocessing performed
- Any known biases or missing values

## 3. Method
- Featurization approach, and why those features
- Baseline (mean prediction or dummy regression)
- Model and cross-validation design (LOO / 5-fold and the rationale)

## 4. Results
- MAE and R² (including a comparison against the baseline)
- Notable hits and large misses

## 5. Discussion
- What feature importance or error analysis revealed
- Whether the results agree with known physics

## 6. Limits and the next move
- Limits in data size, features, and method
- What to try next

⚠️ Common stumbling blocks

Once you start working, many people fall into the same traps. Knowing them ahead of time makes them easier to avoid.

1. Data leakage

If you apply preprocessing (scaling or feature selection) to all the data outside the cross-validation, test information leaks into training and scores look better than the true skill. Always fit scaling and feature selection on the training side of each split only.

2. A single train/test split on small data

With only a few dozen entries, one 80/20 split leaves just a handful in the test set and the evaluation swings wildly. When data is scarce, use Leave-One-Out or 5-fold cross-validation so every entry is tested exactly once.

3. Overconfidence in extrapolation

Tree models such as random forests cannot predict outside the range of the training data. Even the reference implementation badly missed MgO (measured 7.80 eV), predicting around 3 eV. Do not over-trust predictions for samples that lie beyond the training range.

4. Features detached from physics

Even when predictions land, if you cannot give the features a physical meaning, the model may have merely latched onto a coincidental correlation. Aim to explain in a sentence why a feature works — as with the reference implementation's electronegativity difference (a proxy for ionicity).

5. Reporting only the successes

Showing only the samples you got right leads you to overrate the model. Just as the reference implementation deliberately showed the MgO failure, honestly writing up the misses and the limits actually makes for a more trustworthy report.

🚀 Next steps

Once you have finished a full loop, move on to an application close to your interest, or deepen your grasp of evaluation.

Battery Materials MI Application Practice data-driven materials design around next-generation batteries.
Catalyst Materials MI Application Learn how to theoretically design and screen highly active catalysts.
Drug Discovery MI Application Follow the flow of AI-driven drug discovery, from molecular design to ADMET prediction.
Introduction to Robotic Lab Automation Grasp the big picture of closed-loop materials development with autonomous experiments.
Introduction to Model Evaluation (ML dojo) Deepen your grasp of cross-validation, metric choice, and spotting overfitting, systematically.

🎓 Capstone Mini-Project: Build a Property-Prediction Model on Your Own Data