Evaluation and Control
You have to understand that there are two stages, evaluation and control.
Evaluation
Estimate/predict the expected rewards from following a given policy.
Here we don’t care about finding the best Policy, we just want to know the value.
Control
Optimization: find the best policy
Next
Between these two catories, we also have model-free methods.