Artificial General Intelligence

Idea spearheaded by Yann LeCun: https://openreview.net/pdf?id=BZ5a1r-kVsf

See this paper

Talks about how one of the dangers is AI being drawn to Power.

Barriers to reaching AGI

The biggest philosophical question is about how we encode values into an AI.

1. ?
2. $k<n$

Serena brought up this point about $k<n$. That there are too many variables to keep track of in the real world. This relates to Software 2.0. Before, we wrote specifications for our own software (I learned in SE212, so it behaves in ways we expect.

However, now it is an AI.

Optimal solution to life

AGI will solve how we optimally live life. That is why I am interested. But optimality is dependent on the reward question. This is the Meta-Problem.

It is not how to optimally live life, it is what is the reward function!!!

This is why the answer to life is 42. The optimal way to live life is 42.

Think of 42 as just a particular policy. But what are you optimizing towards? That is the question.